Splunk Search

Splunk Health Check (Warning, Info and N/A)

vino06
New Member

Hi Guys,

Good Day!

Regarding on our Splunk servers, we've performed a health check and we found some warning, info and n/a status. Just want to ask what this are and how this result impacts on our application. Please see below screenshot.

Thank you!
Kevin

0 Karma

mattymo
Splunk Employee
Splunk Employee

Hi Kevin!

The actions you should take are dependent on which nodes are involved and require some knowledge about your environment, but I'll do my best to set you in the right direction here.

ULIMITS & THP
For the System and Environment warnings, it is telling you that THP and ulimits are not optimally set for a Splunk Enterprise instance. I would only be worried about correcting this in your core Splunk nodes (Indexers, Search Heads, etc....anything other than Universal Forwarders, really).

Here is some documentation on THP and Ulimits. How you these depends on your system, so work with your sysadmins to ensure they are set persistently.

About Ulimit - https://docs.splunk.com/Documentation/Splunk/6.6.2/Troubleshooting/ulimitErrors
Example How to adjust - https://www.tecmint.com/increase-set-open-file-limits-in-linux/
* note in the versions of RHEL i have played with lately, setting the ulimits on boot-start can be tricky, consult your vendor docs if necessary.

About THP - https://docs.splunk.com/Documentation/Splunk/6.6.2/ReleaseNotes/SplunkandTHP
Example How to Disable THP in Centos 7 - https://newbiedba.wordpress.com/2015/09/07/disabling-transparent-huge-pages-in-centos-7-x/

Missing Forwarders

Missing forwarders is simply forwarders that have not been seen in the last 15 minutes by your indexers. You can see more on this in Forwarder Management dashboard and can be resolved by rebuilding your forwarder lookup if need be. This may or may not be impacting you depending on whether they are decommissioned servers, etc. You will need to

Skipped Searches

This is something you will need to look at in your search heads. You can use the Monitoring Console to analyze your search performance. There is a good break down on what is being skipped under Settings > Monitoring Console > Search > Search Activity: Instance

- MattyMo
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...