Splunk Search

IndexScopedSearch Error

sranga
Path Finder

Hi

We recently upgraded our Splunk instance from 4.0.10 to 4.1.4. After the upgrade we are seeing the following error being generated from one of the charts using a summary-indexed query:

Error in 'IndexScopedSearch': The search failed. More than 125000 events found at time 1286150400.

The query is of the form:

index=summary s_name="blah" | stats count as s_count by orig_host, field1, _time | 
timechart sum(s_count) by orig_host  

This chart worked fine in 4.0.10. Thanks for your help.

Ranga

0 Karma

Lowell
Super Champion

Looks like you are producing too many events on a single timestamp within your summary index saved search.

1286150400 is 2010-10-04 00:00.00

Are you producing your summary index events with sistats? If so, you should look very carefully at which stats functions you are using. You can actually end up with more events in your summary index than you are summarizing depending on your field/function combinations.

Update:

If you haven't found this already, the following is helpful information in understanding the issue at hand. (I'm fairly certain this is the same issue, but I could be wrong.)

http://answers.splunk.com/questions/303/whats-max-events-i-can-have-timestamped-with-a-particular-se...

0 Karma

Lowell
Super Champion

@dmlee, you may want to open a new question. This question is specifically referring to a change in behavior after an upgrade; but there is a more generic question here: How to avoid 125000 events limit in my summary index? If you ask a question like that you me able to get some additional attention. I don't have a good answer, especially without seeing the related searches. I added a link of interest to my answer that may be helpful too.

0 Karma

Lowell
Super Champion

@sranga, sorry for getting back so late, I must have missed your comment. While you can use addinfo and collect and all that, you don't have to, and that's not what I'm suggesting that you change. I'm simply pointing out that you can change your search to use | stats instead of using | sistats the si-prefixed search operators sometimes give you extra value add and generally lets you do less thinking upfront, but I've found that blindly choosing sistats over stats can lead to other problems if you aren't paying attention to how much output it creates.

0 Karma

dmlee
Communicator

we also have this problem, is there any way to extend the limition ? because we can't find other way to reduce events on a single timestamp.

0 Karma

sranga
Path Finder

Thanks. If I don't use sistats, how do I summary index my data? Are you suggesting that I manually configure the summary index using addinfo and collect?

0 Karma

Lowell
Super Champion

For a simple count like that, I'm not sure that sistats buys you anything that plain stats wouldn't do. You do have to watch out for how you name your fields, but that's not a big deal. It would also be good to confirm that your scheduled interval and search time range aren't overlapping or anything like that. That could certainly lead to too many summary events as well.

0 Karma

sranga
Path Finder

Thanks. Yes, I use sistats. The query specified in the question is the one I use in my summary index and it is of the form: index=summary s_name="blah" | sistats count as s_count by orig_host, field1, _time

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...