Splunk Search

How to remove incorrectly timestamped events from index?

splunkIT
Splunk Employee
Splunk Employee

Got some indexed events that were incorrectly timestamped, like set to 20 years into the feature, and would like to know if the DELETE command would help in removing these bad events from the index.

Concern with these bad events, if not removed:
1. bucket will not roll to frozen/archiving. Currently, the frozen policy is 1-year.
2. performance impact, by having these events around

Tags (1)

martin_mueller
SplunkTrust
SplunkTrust

The delete command will not remove data from the index properly, it will just exclude the events from future searches.

To avoid these wildly wrong timestamps from getting into your index you could look at reducing your MAX_DAYS_HENCE setting in props.conf, by default it's just a couple of days. Anything much larger rarely makes sense.

As for rolling the affected bucket to frozen, it might indeed not roll it on its own because it contains extremely new events far from the future. You could manually roll the bucket though, once you're certain the other correctly timestamped events are due for a freeze.

martin_mueller
SplunkTrust
SplunkTrust

Warm and cold buckets' directory names contain the newest and oldest event timestamp in the bucket - in your very specific case of events indexed 20 years into the future that's enough to find the buckets containing badly timestamped events.

Another lazy way would be to wait a year for the natural roll to frozen, and pick out those that should have rolled but didn't.

0 Karma

splunkIT
Splunk Employee
Splunk Employee

Is there a way to determine which buckets are affected by the bad events?

0 Karma
Get Updates on the Splunk Community!

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...