All Apps and Add-ons

Retention Limits on a index question

daniel333
Builder

All,

I am on a roll for questions today. Just learning Firebrigade here, cool app. But I don't understand how it's possible that I can have data in an index older than the max age of the index?

I have a 30 day max on my activemq index but we can see data 48 days old in there. Shouldn't splunk dump that? Is there a way to force it to dump?

alt text

0 Karma

sowings
Splunk Employee
Splunk Employee

The span of a bucket is defined as the time between the oldest event (in your case, 48 days) and the youngest event in that bucket. The span of the index in its entirety is the oldest event from any bucket to the youngest event in any bucket. A bucket's "age" is calculated only based upon the youngest event in the bucket. It's entirely possible (and this is what's happening in your case) that a bucket's data can span multiple days. Since the time-based expulsion of data buckets is based upon the bucket's age (and therefore the newest event in that bucket) this bucket isn't yet "old enough".

You can get a bit more info about this bucket with the "Retention > Bucket Age vs. Age Limit" dashboard. You'll see the ten oldest buckets (those next on the chopping block) including an indication of the oldest event in that bucket. If you don't see your March 18th dates in that list, then that bucket isn't yet old enough (and in fact may have future dates!).

As for the question of forcing the data to dump; not really. You're only at 8% of capacity on that index, so Splunk own built-in policies won't delete it. However, you could go under the hood and remove individual buckets if you wanted. Given the potential for "warranty-voiding" activity, I'm loath to provide more detail than that!

0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

ICYMI - Check out the latest releases of Splunk Edge Processor

Splunk is pleased to announce the latest enhancements to Splunk Edge Processor.  HEC Receiver authorization ...

Introducing the 2024 SplunkTrust!

Hello, Splunk Community! We are beyond thrilled to announce our newest group of SplunkTrust members!  The ...