Getting Data In

Rotating Data to Frozen After Time Period

andrewkenth
Communicator

What is the best way to rotate events into Frozen OR delete events that are older than 18 months?

I can think of a few off the top of my head but what is the best or indented way to do this?

1) indexes.conf?
frozenTimePeriodInSecs seems to require a script? Why not just to the frozen dir identified in settings?

2) Run delete searches w/ a timespan?

3) A better way?

Tags (2)

Adrian
Path Finder

As long as you specify coldToFrozenDir in your indexes.conf you shouldn't have any problems using frozenTimePeriodInSecs and set it to 1555200 (seconds in 18 days)

Here is the documentation:
http://docs.splunk.com/Documentation/Splunk/6.0.1/Indexer/Configureindexstorage

and here:
http://docs.splunk.com/Documentation/Splunk/6.0.1/Indexer/Setaretirementandarchivingpolicy

Adrian
Path Finder

Good point... I was assuming retention was necessary when in fact it is not a requirement.

0 Karma

jbsplunk
Splunk Employee
Splunk Employee

Data is frozen irrespective of it's location if the threshold for the setting is reached. Data can exist in the homePath and still be frozen. If you were to create a test index with a very short retention period (1h, for instance), it's very likely that as soon as a bucket rolls to warm, it'll disappear to bring the index retention policy back into compliance.

0 Karma

aelliott
Motivator

It's actually in the indexes.conf documentation:

maxHotSpanSecs

http://docs.splunk.com/Documentation/Splunk/6.0.1/admin/Indexesconf

0 Karma

andrewkenth
Communicator

Thank you! How do I configure the duration for the Hot/Warm to Cold bucket move?

0 Karma

aelliott
Motivator

however it will not be rolled into frozen until it has completed it's journey into cold, which would have to be set to 18 days as well, not to mention the hot/warm time. So the data will remain for 36 days in this scenario with the option of restoring the frozen 18 days of data at any given point

0 Karma

aelliott
Motivator

The information in this post may assist you, Basically you can setup how long something should be in a specified bucket, You can say that something can stay in cold for 18 months and then it will automatically be deleted (if no frozen script is specified), However the data will be as old as the Hot/warm time as well, before starting it's journey into cold.

http://answers.splunk.com/answers/114896/splunk-index-retention-based-on-retention-period-only-not-s...

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...