Getting Data In

Indexed data not erased

Hazel
Communicator

Hello,

Our indexes seem to be taking up too much disk space so rather than just moving them i'd like to look at the best way to change our approach.

According to this page, http://www.splunk.com/base/Documentation/latest/admin/SetARetirementAndArchivingPolicy data will be automatically frozen after approx 6 years:

To remove data beyond a specified age, set frozenTimePeriodinSecs in indexes.conf to the number of seconds to elapse before the data gets erased. The default value is 188697600 seconds, or approximately 6 years.

However, if I go on to the Manager and look at my indexer, it is showing information from the os index dating back to 8 Aug 2003 12:29:17 as the Earliest Event, which is beyond 6 years. How can I check if this property is working as it doesn't seem to be to me?

Thanks! Hazel

Tags (2)

southeringtonp
Motivator

Splunk won't age out buckets to frozen until all events in that bucket are older than frozenTimePeriodinSecs.

In the search app, take a look at Status->Index Activity->Index Health.

Also take a look at the | dbinspect search command.

southeringtonp
Motivator

Yes, that's correct.

0 Karma

Hazel
Communicator

(I think you mean all events in the bucket here not all events in the index?)

0 Karma

Simeon
Splunk Employee
Splunk Employee

Splunk will delete based upon the latest event contained in a bucket. For this reason, you may not see exact timing for deletion of data if there is a large overlap in timing. To inspect the span of a db/bucket, you can use the dbinspect command to see the time range of each bucket.

It's possible that you have data in a bucket that is before and later than the deletion time. Since we delete based on the latest (most recent) event in a bucket, you may still see older events in an index.

Hazel
Communicator

Thanks I got it. I can see buckets that have earliest time as 2003 but latest time as 2010. So is there no easy way to clear out my old data or make this command work?

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...