Splunk Search

Streamed search execute failed because: Error in 'surrounding': Too many events (> 10000) in a single second.

splunk_zen
Builder

Even though I have overwritten what I believe is this limit in limits.conf,
btool is showing,

[show_source]
max_count = 50000
distributed_search_limit = 30000
distributed = true

The error message displays 10k rather than 50k.
Is this a bug as in the parameter is not being respected, or a bug as in the message not displaying the value Splunk is enforcing ?

Any recommendation on how to allow to check the source for

0 Karma
1 Solution

jkat54
SplunkTrust
SplunkTrust

1st, know your limits:

http://docs.splunk.com/Documentation/Splunk/6.2.3/admin/Limitsconf

limits.conf settings and DISTRIBUTED SEARCH
Unlike most settings which affect searches, limits.conf settings are not
provided by the search head to be used by the search peers.
This means that if
you need to alter search-affecting limits in a distributed environment, typically
you will need to modify these settings on the relevant peers and search head for
consistent results.

2nd, tell us your architecture. If you only have 1 server, my answer above is null and void.

3rd, as mentioned there is a configuration file precedence issue possibly. See the following:
http://docs.splunk.com/Documentation/Splunk/6.2.0/admin/Wheretofindtheconfigurationfiles

View solution in original post

jkat54
SplunkTrust
SplunkTrust

1st, know your limits:

http://docs.splunk.com/Documentation/Splunk/6.2.3/admin/Limitsconf

limits.conf settings and DISTRIBUTED SEARCH
Unlike most settings which affect searches, limits.conf settings are not
provided by the search head to be used by the search peers.
This means that if
you need to alter search-affecting limits in a distributed environment, typically
you will need to modify these settings on the relevant peers and search head for
consistent results.

2nd, tell us your architecture. If you only have 1 server, my answer above is null and void.

3rd, as mentioned there is a configuration file precedence issue possibly. See the following:
http://docs.splunk.com/Documentation/Splunk/6.2.0/admin/Wheretofindtheconfigurationfiles

splunk_zen
Builder

Thanks jkat54.
That makes sense, will push these changes to the Indexer cluster then.
There's no configuration file precedence issue as confirmed by btool, set this up in a specific App to exclusively target limits.conf (thus taking predence over system/{default,local})

jkat54
SplunkTrust
SplunkTrust

number 1 should help you then! Thanks for marking the answer.

0 Karma

renjith_nair
Legend

Try to do that in local/limits.conf and restart splunk after that if not done already

Happy Splunking!
0 Karma
Get Updates on the Splunk Community!

Splunk Custom Visualizations App End of Life

The Splunk Custom Visualizations apps End of Life for SimpleXML will reach end of support on Dec 21, 2024, ...

Introducing Splunk Enterprise 9.2

WATCH HERE! Watch this Tech Talk to learn about the latest features and enhancements shipped in the new Splunk ...

Adoption of RUM and APM at Splunk

    Unleash the power of Splunk Observability   Watch Now In this can't miss Tech Talk! The Splunk Growth ...