Splunk Search

Is max_searches_per_cpu at 6 a good set-up?

danielbb
Motivator

On our primary search head max_searches_per_cpu is set to 6. I wonder if it’s a good effective set-up. Where can I find how many searches each SH cpu is actually running?

0 Karma

richgalloway
SplunkTrust
SplunkTrust

The Monitoring Console can show how many searches are running. Select "Scheduler Activity: Instance" from the search menu to see how many searches run at a time.

It difficult to say if 6 is a good number since we know nothing about your server environment. Considering the default value is 2, 6 seems a bit high. Keep in mind that each search not only uses a CPU on the search head, but also uses a CPU on every indexer.

---
If this reply helps you, Karma would be appreciated.

danielbb
Motivator

Thank you @richgalloway - is there a way to determine how effective the value of 6 is?

0 Karma

richgalloway
SplunkTrust
SplunkTrust

If you're not seeing skipped searches and the load on your SH and indexers is tolerable then 6 may be OK. It's possible not running that many searches. The MC will tell you.

---
If this reply helps you, Karma would be appreciated.

danielbb
Motivator

@richgalloway - the reason we check these values is because we see skipped searches. Sorry for not being clear...

0 Karma

richgalloway
SplunkTrust
SplunkTrust

You should see messages in the logs or MC about why searches are being skipped. Usually, it's because there are too many searches running, but that shouldn't be your problem with 6 searches per CPU (although it could be). Make sure all of your searches have a Schedule Window set ("auto" or some non-zero value). Try to spread scheduled searches around the clock rather than have them run at the top of the hour.

It's also possible searches are skipped because the same search is still running. If that's the case, tune the search to improve performance or increase the schedule interval.

---
If this reply helps you, Karma would be appreciated.

danielbb
Motivator

Schedule Window set to "auto" !!! I'm with you on that @richgalloway .

0 Karma

kamlesh_vaghela
SplunkTrust
SplunkTrust

@danielbb

Try below search for list of searches in particular period. I hope it will help you.

index=_audit action="search" search=* NOT user="splunk-system-user" | table host user search * 
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...