how is age of an event calculated before the bucket is frozen

[pavankumarh]

hi, when we set frozenTimePeriodInSecs=30 days for an index. i have read through the documentation and forums that the bucket moves to frozen state when all the data in the given bucket is more than 30 days old. but i need some more clarity. basically the condition i think is one of the below for the latest event in the bucket.

_time > frozenTimePeriodInSecs
OR
_indextime >frozenTimePeriodInSecs

Please let me know which time controls the retention.

[lukejadamec · Accepted solution]

It's _time
See this post. It provides a very good review:
https://wiki.splunk.com/Deploy:SplunkBucketRetentionTimestampsAndYou

[pavankumarh]

Thanks Luke.. The explanation in wiki link was a bit complex and took time to understand but it certainly supports your answer.

I was running two parallel daily reports - one based on _time and the other based on _indextime to verify if my retention values were working as expected. But both confusing me.
Now i will concentrate on _time based one for my further analysis.

Thanks Again.

[lukejadamec]

It is complex, but necessarily so because while you may want old data, you probably most certainly want current data.

What may not be evident in the wiki post is the indexing very old logs and logs 'in the future'.

Let's say for example you have logs that are older than the MAX_DAYS_AGO parameter from props.conf (default 2000 days) - all events older than the MAX_DAYS_AGO will have the _time value of the last acceptable timestamp in that log file, and if all events are older than MAX_DAYS_AGO, then all events will have the current (index time) timestamp for _time.

So, in addition to knowing that Freezing data is based on _time, you must also understand that it is based on the most recent time in the buckets (restart Splunkd = new buckets), and that indexing very old logs may give you event times (_time) that are not what you expected (MAX_DAYS_AGO defaults to 2000 days).

[richgalloway]

I'm 95% certain it's _time.