Splunk Search

Is max_searches_per_cpu at 6 a good set-up?

danielbb
Motivator

On our primary search head max_searches_per_cpu is set to 6. I wonder if it’s a good effective set-up. Where can I find how many searches each SH cpu is actually running?

0 Karma

richgalloway
SplunkTrust
SplunkTrust

The Monitoring Console can show how many searches are running. Select "Scheduler Activity: Instance" from the search menu to see how many searches run at a time.

It difficult to say if 6 is a good number since we know nothing about your server environment. Considering the default value is 2, 6 seems a bit high. Keep in mind that each search not only uses a CPU on the search head, but also uses a CPU on every indexer.

---
If this reply helps you, Karma would be appreciated.

danielbb
Motivator

Thank you @richgalloway - is there a way to determine how effective the value of 6 is?

0 Karma

richgalloway
SplunkTrust
SplunkTrust

If you're not seeing skipped searches and the load on your SH and indexers is tolerable then 6 may be OK. It's possible not running that many searches. The MC will tell you.

---
If this reply helps you, Karma would be appreciated.

danielbb
Motivator

@richgalloway - the reason we check these values is because we see skipped searches. Sorry for not being clear...

0 Karma

richgalloway
SplunkTrust
SplunkTrust

You should see messages in the logs or MC about why searches are being skipped. Usually, it's because there are too many searches running, but that shouldn't be your problem with 6 searches per CPU (although it could be). Make sure all of your searches have a Schedule Window set ("auto" or some non-zero value). Try to spread scheduled searches around the clock rather than have them run at the top of the hour.

It's also possible searches are skipped because the same search is still running. If that's the case, tune the search to improve performance or increase the schedule interval.

---
If this reply helps you, Karma would be appreciated.

danielbb
Motivator

Schedule Window set to "auto" !!! I'm with you on that @richgalloway .

0 Karma

kamlesh_vaghela
SplunkTrust
SplunkTrust

@danielbb

Try below search for list of searches in particular period. I hope it will help you.

index=_audit action="search" search=* NOT user="splunk-system-user" | table host user search * 
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

Splunk is officially part of Cisco

Revolutionizing how our customers build resilience across their entire digital footprint.   Splunk ...

Splunk APM & RUM | Planned Maintenance March 26 - March 28, 2024

There will be planned maintenance for Splunk APM and RUM between March 26, 2024 and March 28, 2024 as ...