Summary index limits events to 50000

[bharathkumarnec]

Hello All,

I am facing an issue with the summary indexing, always the data in the summary index is limited to 50000 events but when i run the query manually i get more than them actually doubt the above count.

I have verified the size of my summary index and dispatch.max_count value and both are more to accept the acutal numbers.

But events are getting trimmed to 50000 in summary index. I have checked both verbose and fast mode.

Kindly help me out in where i need to check??

Regards,
BK

[mayurr98 · Accepted solution]

There's by default a limit of 10,000 events that will get summary indexed from each run of a scheduled search. This default limit will actually be removed in most cases in the next maintenance release (4.1.2). For now, here's a couple of workarounds.

1) in savedsearches.conf for the app that your search belongs to, under the stanza for your scheduled searches, add dispatch.max_count=100000 (or whatever limit you want) ALSO, in etc/system/local/limits.conf (create it if it doesn't exist), under the [scheduler] stanza, set max_action_results=100000 (or a limit of your choosing).

OR

2) instead of setting enabling the summary indexing action of the saved searches, explicitly add a " | collect" to the end of your saved searches. This will change the search itself to directly populate your summary index. (instead of the default behavior, which is that the scheduler reads the result of your search and then populates the summary index from that result)

let me know if this helps !

[rmanrique]

I followed your first recommendation and it hasn't worked for me. In my case I have to distributed search, I have one SH and two IDX. I created the limits.conf file in SH.

Should I do it in IDXs?