Monitoring Splunk

How to monitor linux server cpu usage from a windows splunk server?

friscos
Explorer

Hi,

I have forwarders installed on the linux servers and my splunk server is installed on windows machine.

I am looking for a documentation on how to configure the splunk to monitor the CPU utilization of linux servers.

I have installed Splunk App for Unix on Splunk Server but it doesn't recognize the unix servers where forwarders were installed. I am assuming that there may be a configuration to add them.

Could someone help me please.

Tags (1)
0 Karma

friscos
Explorer

Thanks Muebel for your response. I do not see any errors in splunkd.log.

All log files are being monitored correctly on splunk enterprise but somehow do not see CPU information in 'Splunk App for Unix'.

0 Karma

friscos
Explorer

Yes splunkd is running as a root. I see all the logs being updated in the splunk enterprise but do not see anything in 'Splunk App for Unix'.
I do not see any errors in splunkd.log.

0 Karma

muebel
SplunkTrust
SplunkTrust

Hi friscos, You'll want to check out the add-on for linux and unix, and install that on the nix forwarder https://splunkbase.splunk.com/app/833/

This app contains scripted inputs to collect CPU utilization, as well as other hardware performance/availability metrics.

You'll potentially need to create the "os" index (default index for these inputs) or otherwise override the index config in the local inputs.conf for the app.

Please let me know if this answers your question!

0 Karma

friscos
Explorer

Hi,

I have installed 'Splunk App for Unix' on the Splunk Enterprise (Windows OS) and installed 'Splunk Add on for unix and linux' on the linux forwarders.
Followed the document and created an inputs.conf under /apps/splunkforwarder/etc/apps/Splunk_TA_nix/local and added the below content:

[script://./bin/cpu.sh]
sourcetype = cpu
source = cpu
interval = 30
index = os
disabled = 0

Restarted both the forwarder and Splunk enterprise, Do not see any CPU usage on Splunk Enterprise though the host names are listed.

When i try to manually enable the cpu on forwarder, i am getting the below error.

[root@xxxxxxxxxxx bin]# ./setup.sh --interval cpu.sh 120
./setup.sh: line 50: /bin/splunk: No such file or directory
./setup.sh: line 834: /bin/splunk: No such file or directory

authenticated to

setting cpu.sh interval to 120
./setup.sh: line 110: /bin/splunk: No such file or directory
update failed

Did I miss any steps here?

0 Karma

muebel
SplunkTrust
SplunkTrust

Hi friscos,

Do you see any other logs making it from the forwarder into the indexer, in particular, at least the _internal (splunkd) type logs?

Other than that, it looks like its having a problem finding dependencies when you run it manually. This could be related to some path related variables that are missing, but are set when the input normally runs as a scripted input.

I'd check all the other splunk configs first (outputs to make sure any events at all are making it to the indexer)

0 Karma

friscos
Explorer

Yes, I configured around 10 log files on the forwarder and I see all of them in the Splunk Enterprise.

You are right, SPLUNK_HOME was not set and after setting the splunk_home, I was able to enable the cpu.

./setup.sh --list-all
1) /apps/splunkforwarder/etc/apps/Splunk_TA_nix/bin/cpu.sh
enabled: *** disabled: interval: 30

I could perform search on the forwarder log file but do not see CPU usage on 'Splunk App for Unix'. I see the
forwarder hostname listed under Group.

which splunk log file that i should look for any errors?

0 Karma

muebel
SplunkTrust
SplunkTrust

splunkd.log on the linux box. Is the forwarder running as root?

0 Karma

friscos
Explorer

Yes splunkd is running as root. I do not see any errors in splunkd but 'Splunk App for Unix' is not receiving any input in splunk enterprise.

enabling apps/splunkforwarder/etc/apps/Splunk_TA_nix/bin/cpu.sh
enable successful

anything else that i should look for?

0 Karma

friscos
Explorer

Thank you very much. I installed the add-on on the splunk server and didn't know that it needs to be installed on the forwarder aswell.

I found the documentation. will install it and update here.. Thanks Again.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...