Splunk Search

Is there a method for rolling data completely out of an idex based on its age?

cdavidy
Explorer

I have a need to automatically roll data completely out of an index so that it's no longer searchable after a number of days. Is this possible? how do I go about it?

Tags (1)

gkanapathy
Splunk Employee
Splunk Employee

You can set frozenTimePeriodInSecs, but it does not necessarily guarantee that data will be removed when it reaches that age. What it does specify is that data may be rolled out when it hits that age, provided that everything in its data bucket is also aged enough to be rolled out.

You can control to some degree by setting maxHotSpanSecs, but this setting can have significant impact on search performance and changing it probably requires changing other index configurations, and generally should not be done without official Splunk recommendations.

0 Karma

BunnyHop
Contributor

This is the answer to do the rolling automatically.

http://answers.splunk.com/questions/664/how-can-i-trigger-migration-of-buckets-from-warm-to-cold

Keep in mind that both hot and warm are still searchable. If you want to search the cold bucket, you can specify that.

gkanapathy
Splunk Employee
Splunk Employee

In 4.x, there is no functional distinction in search between data in hot or cold buckets, i.e., searches will look in cold automatically as appropriate.

0 Karma

Simeon
Splunk Employee
Splunk Employee

You can set the retention period of an index via the indexes.conf file:

http://www.splunk.com/base/Documentation/latest/admin/Indexesconf

frozenTimePeriodInSecs = <integer>
* Number of seconds after which indexed data rolls to frozen.
* If you do not specify a coldToFrozenScript, this data is erased.
* IMPORTANT: Every event in the DB must be older than frozenTimePeriodInSecs before it will roll.
* frozenTimePeriodInSecs will be frozen the next time splunkd checks.
* Defaults to 188697600.

You should apply the above value within your specific index stanza. By default, frozen implies that data is deleted.

Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...