Knowledge Management

Summary index limits events to 50000

bharathkumarnec
Contributor

Hello All,

I am facing an issue with the summary indexing, always the data in the summary index is limited to 50000 events but when i run the query manually i get more than them actually doubt the above count.

I have verified the size of my summary index and dispatch.max_count value and both are more to accept the acutal numbers.

But events are getting trimmed to 50000 in summary index. I have checked both verbose and fast mode.

Kindly help me out in where i need to check??

Regards,
BK

0 Karma
1 Solution

mayurr98
Super Champion

There's by default a limit of 10,000 events that will get summary indexed from each run of a scheduled search. This default limit will actually be removed in most cases in the next maintenance release (4.1.2). For now, here's a couple of workarounds.

1) in savedsearches.conf for the app that your search belongs to, under the stanza for your scheduled searches, add dispatch.max_count=100000 (or whatever limit you want) ALSO, in etc/system/local/limits.conf (create it if it doesn't exist), under the [scheduler] stanza, set max_action_results=100000 (or a limit of your choosing).

OR

2) instead of setting enabling the summary indexing action of the saved searches, explicitly add a " | collect" to the end of your saved searches. This will change the search itself to directly populate your summary index. (instead of the default behavior, which is that the scheduler reads the result of your search and then populates the summary index from that result)

let me know if this helps !

View solution in original post

mayurr98
Super Champion

There's by default a limit of 10,000 events that will get summary indexed from each run of a scheduled search. This default limit will actually be removed in most cases in the next maintenance release (4.1.2). For now, here's a couple of workarounds.

1) in savedsearches.conf for the app that your search belongs to, under the stanza for your scheduled searches, add dispatch.max_count=100000 (or whatever limit you want) ALSO, in etc/system/local/limits.conf (create it if it doesn't exist), under the [scheduler] stanza, set max_action_results=100000 (or a limit of your choosing).

OR

2) instead of setting enabling the summary indexing action of the saved searches, explicitly add a " | collect" to the end of your saved searches. This will change the search itself to directly populate your summary index. (instead of the default behavior, which is that the scheduler reads the result of your search and then populates the summary index from that result)

let me know if this helps !

rmanrique
Path Finder

I followed your first recommendation and it hasn't worked for me. In my case I have to distributed search, I have one SH and two IDX. I created the limits.conf file in SH.

Should I do it in IDXs?

0 Karma
Get Updates on the Splunk Community!

Database Performance Sidebar Panel Now on APM Database Query Performance & Service ...

We’ve streamlined the troubleshooting experience for database-related service issues by adding a database ...

IM Landing Page Filter - Now Available

We’ve added the capability for you to filter across the summary details on the main Infrastructure Monitoring ...

Dynamic Links from Alerts to IM Navigators - New in Observability Cloud

Splunk continues to improve the troubleshooting experience in Observability Cloud with this latest enhancement ...