Monitoring Splunk

Data Age / Frozen Age (days): How to delete the data older than frozen time?

vumanhtai
Path Finder

Hi Splunk Team
in splunk monitor consonle i see that Data Age vs Frozen Age (days) like this: 1996/450
as per my understanding event older than frozen age (450) will be delete
but in hear the data age (1996)

Please help me explain this issue and how to delete the data older than Frozen time.

Thank you

Labels (1)
Tags (2)
0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @vumanhtai,
events are stored in buckets.
When a bucket passes from Hot to Warm state, the values of the oldest and newest events are fixed in the bucket.
So bucket will be removed when the newest event timestamp will exceed the retention time, so if in an index you have few events, you could have in the same buckets events of 3 years ago and events of last year, so the bucket isn't deleted: you have to wait for the newest event exceeding the retention time.

To avoid this, you could create different rules for your indexes: indexes with few events are rolled from Hot to Warm state more frequently than other indexes with many events.

See in https://docs.splunk.com/Documentation/Splunk/8.0.0/Admin/Indexesconf which configuration could be the best for your indexes.

Ciao.
Giuseppe

saschakoerner
Explorer

Hi, 

if index is configured with maxHotSpanSecs = 86400 I thought that index buckets will be rolled every day. But in when I check index details: instance I have the following: 

Data Age 422 <-> Frozen Age (days): 90

Any ideas on that ?

Kind regards

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @saschakoerner,

for configuring retenton, you have to setup the "frozenTimePeriodInSecs" parameter for each index.

Anyway, as i said, you could have events that exceed the retention period because they are in a bucket that contains also events that don't exceed the retention period.

About maxHotSpanSecs, if you read the answer at https://community.splunk.com/t5/Splunk-Search/Difference-between-maxHotIdleSecs-and-maxHotSpanSecs/m... you can understand the different options that you can use, anyway this parameter isn't related to the frozed age.

Ciao.

Giuseppe

saschakoerner
Explorer

Hi @gcusello , 

thanks for your answer. 

we have set frozenTimePeriodInSecs globally in the indexes.conf to 7776000 (=90 days) 

At the specific index we have set maxHotSpanSecs to 86400.

So when I am reading the link and the documentation in my understanding index is rolling the buckets every day for a max time of 90 days.  The oldest event could only be 91 days old.
Do I make a mistake ?

splunk btool indexes list --debug "indexname" gives the configured parameters back. 

/appl/splunk/etc/slave-apps/bit_all_indexes/local/indexes.conf frozenTimePeriodInSecs = 7776000
/appl/splunk/etc/slave-apps/bit_all_indexes/local/indexes.conf maxHotSpanSecs = 86400

Regards

0 Karma

PickleRick
SplunkTrust
SplunkTrust

Use the dbinspect command over all time to check each bucket's time range. My personal guess is that you have a so-called quarantine bucket which holds all "too old" events as well as those supposedly from the future. Splunk has one bucket for such events so they don't interfere with "normal" ones. Such bucket can contain events from 420 days ago as well as relatively recent (or even from the future). And it will not be rolled to frozen since _latest_ event from this bucket is not old enough.

gcusello
SplunkTrust
SplunkTrust

Hi @saschakoerner,

I don't like your solution because in this way you have many small buckets (probably too!) and performaces could be affected by this problem.

As descricted in the above answer, leave parameters to default, the only parameter to setup is frozenTimePeriodInSecs .

Ciao.

Giuseppe

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...