Getting Data In

IndexScopedSearch deleting bad events?

jalfrey
Communicator

I'm getting the following error:
Error in 'IndexScopedSearch': The search failed. More than 1000000 events found at time 1390985456.

I found this existing question:
http://answers.splunk.com/answers/76392/how-to-delete-a-huge-number-of-old-events-from-the-test-data...
as well as this one
http://answers.splunk.com/answers/119467/indexscopedsearch-error-while-searching
as well as this one
http://answers.splunk.com/answers/3397/indexscopedsearch-error-details

I tried removing the db directories and restarting splunk. Still getting same error. I tried this command:
_time="1389689456" | delete

Does anyone know how to remove the bad data?

0 Karma

wrangler2x
Motivator

This may have nothing to do with indexed data corruption. If you had something that could generate 1,000 log events in a second, and if your timestamp is accurate only to the second, then all of those log entries would have the same epoch time (which is given in the error message for the frame of time where > 1,000,000 events have been seen). The epoch time is accurate to the millisecond, so in a single second you have 1,000 epoch times but only one is being used for all the events begin logged in any given second. Windows iis logs are by default accurate to the second. But you can google for your version and see about turning on millisecond time stamp. Whatever logs you are using, that's what I'd try.

If this is being caused by corrupted data, there is quite a bit of information about dealing with that here: http://wiki.splunk.com/Community:PostCrashFsckRepair

0 Karma

wrangler2x
Motivator
0 Karma
Get Updates on the Splunk Community!

Routing logs with Splunk OTel Collector for Kubernetes

The Splunk Distribution of the OpenTelemetry (OTel) Collector is a product that provides a way to ingest ...

Welcome to the Splunk Community!

(view in My Videos) We're so glad you're here! The Splunk Community is place to connect, learn, give back, and ...

Tech Talk | Elevating Digital Service Excellence: The Synergy of Splunk RUM & APM

Elevating Digital Service Excellence: The Synergy of Real User Monitoring and Application Performance ...