All Apps and Add-ons

SoS and clusters

techn0gichida
Explorer

The forwarder points to the peer in the cluster per the instructions. How does the SoS technology add-on point itself to the search head?

hexx
Splunk Employee
Splunk Employee

Glad to hear that. Feel free to accept my answer, in that case 🙂

0 Karma

techn0gichida
Explorer

hexx, thanks so much. that last suggestion got it working!

0 Karma

hexx
Splunk Employee
Splunk Employee

Why not enable the input manually in %SPLUNK_HOME%\etc\apps\TA-sos_win\local\inputs.conf then?

[script://.\bin\sospowershell.cmd ps_sos.ps1]
disabled = 0
0 Karma

techn0gichida
Explorer

cannot edit input "./bin/ps_sos.ps1", no input exists with that name
that is the error I get when I use the ps_sos.ps1 with the single quotes removed

0 Karma

techn0gichida
Explorer

still getting the 404 error

0 Karma

hexx
Splunk Employee
Splunk Employee

Ah! In that case, the scripted input you need to enable is 'ps_sos.ps1', not 'ps_sos.sh'.

As the README file of the S.o.S technology add-on for Windows states:

Enable the scripted inputs that collect information for the SoS Splunk CPU/Memory
Usage and Distributed Searches Memory Usage views:

(...)

b) Run the following from a command or PowerShell prompt:

  %SPLUNK_HOME%\bin\splunk _internal call \
  '/servicesNS/nobody/TA-sos/data/inputs/script/.%252Fbin%252Fps_sos.ps1' \
  -post:disabled 0

0 Karma

techn0gichida
Explorer

This is a windows box. It doesn't have grep

0 Karma

hexx
Splunk Employee
Splunk Employee

Please show here the output of:

  • $SPLUNK_HOME/bin/splunk cmd btool inputs list 'script:' --debug | grep -A7 'ps_sos.sh'
  • grep 'ps_sos' $SPLUNK_HOME/var/log/splunk/metrics.log | head -10
0 Karma

techn0gichida
Explorer

It's like the script can't find anything on port 8089

0 Karma

techn0gichida
Explorer

Sorry, the script fails with a 404 error.

0 Karma

techn0gichida
Explorer

I added the input manually and it still isn't showing up. I find the following message in splunkd.log:
splunk-regmon - No enabled entries have been found for regmon or procman in the conf file

0 Karma

hexx
Splunk Employee
Splunk Employee

Sounds like the ps_sos.sh scripted input was not successfully enabled on the forwarder. I would suggest to use "splunk login" and log in as admin before running that command again.

Alternatively, you can enable that input manually in $SPLUNK_HOME/etc/apps/TA-sos/local/inputs.conf.

0 Karma

techn0gichida
Explorer

This script fails with a 401 error:
$SPLUNK_HOME/bin/splunk _internal call '/servicesNS/nobody/TA-sos/data/inputs/script/.%252Fbin%252Fps_sos.sh' -post:disabled 0

0 Karma

techn0gichida
Explorer

I did now and it still isn't showing any SoS data. Like I said previously, I did the "index=sos sourcetype=ps | stats count by host" test per the installation instructions but it isn't returning the name of the server in the list.

0 Karma

hexx
Splunk Employee
Splunk Employee

Have you manually added the forwarder to the "splunk_servers_cache.csv" lookup in $SPLUNK_HOME/etc/apps/sos/lookups on the search-head, as recommended?

0 Karma

techn0gichida
Explorer

When I run the test on the search-head it does not return the server name in the list. So the forwarder is not sending any SoS data to the search-head. Although the forwarder is sending splunk data.

0 Karma

hexx
Splunk Employee
Splunk Employee

I think you're asking "Now that my forwarder is collecting data with the scripted inputs of the S.o.S technology add-on, how do I consult that information in the S.o.S app on the search-head?".

If that is accurate, please consult this Splunk Answer which addresses that scenario.

The short version is: You'll need to manually add your forwarder to the "splunk_servers_cache.csv" lookup.

We have plans to make this an automated step in a future release.

Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...