Splunk Search

Why am I getting "maximum number of historical searches that can be run concurrently. current=10 maximum=10" for real time searches?

medunmeyer
Explorer

I am running Splunk 6.5 , and I have tried many things for hours, but am still getting:

The system is approaching the maximum number of historical searches that can be run concurrently. current=10 maximum=10

I have read all the similar post and followed the tips.

I could run only 10 Real Time searches, thus using the formula:

base_max_searches + #cpus*max_searches_per_cpu

6 + (4*1) =10 I understood why only 10, so I upgraded the search head to 16 cpu.
Then 6 + (16*1) = 22 so I should be able to run 22, but still only 10 real time searches run.
Monitoring console sees the 16 cpus.

I then started working with the limits.conf file creating a file in system/local that only had the changes I changed each of these one at a time with the refreshes and restarts needed.

# the maximum number of concurrent searches per CPU 
max_searches_per_cpu = 2
# the base number of concurrent searches
base_max_searches = 10
# max real-time searches = max_rt_search_multiplier x max historical searches
max_rt_search_multiplier = 2

I still can only run 10 Real time searches - the owner has admin rights.

On another search head, I was getting the 10 concurrent searches error above (4 CPU)
I copied the same limits.conf file to system/local. Restarted Splunk and the same message appeared when loading distributed monitoring console.

Is there something else to do so it sees the additional cpu's and or the settings in the limits.conf file?

Thanks in advance for any help

0 Karma
1 Solution

medunmeyer
Explorer

A couple of things - first it was actually 11 concurrent searches because of this setting

[scheduler]
# the maximum number of searches the scheduler can run, as a percentage
# of the maximum number of concurrent searches 
max_searches_perc  = 50

and settings changes did not work because I missed the [search]

View solution in original post

0 Karma

medunmeyer
Explorer

A couple of things - first it was actually 11 concurrent searches because of this setting

[scheduler]
# the maximum number of searches the scheduler can run, as a percentage
# of the maximum number of concurrent searches 
max_searches_perc  = 50

and settings changes did not work because I missed the [search]

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...