Deployment Architecture

How to set the bucket retention policy for 2 months before automatically deleting the data?

sbenamro
New Member

Hi,

I'd like to set the Bucket time to keep data for 2 months and then to automatically remove it.

how do I do it ?

0 Karma
1 Solution

martin_mueller
SplunkTrust
SplunkTrust

The key settings for short retention times are these two indexes.conf:

frozenTimePeriodInSecs = <nonnegative integer>
maxHotSpanSecs = <positive integer>

The first setting defaults to six years, you'll want to set that to two months so 5270400 (61 days) or however long your months are in seconds. The second setting defaults to 90 days, which is great for six years of retention but not great for two months. A bucket is removed when its youngest event crosses the frozenTimePeriodInSecs, so you would effectively wait five months. Set this to a week or so to actually remove your data close to those two months.

See http://docs.splunk.com/Documentation/Splunk/6.2.2/admin/indexesconf for reference.

View solution in original post

NOUMSSI
Builder

Hi,

Here is an example to show you how to set it:

[default]
maxWarmDBCount = 200
frozenTimePeriodInSecs = 432000
rotatePeriodInSecs = 30
coldToFrozenScript = "$SPLUNK_HOME/bin/python"
"$SPLUNK_HOME/bin/myColdToFrozenScript.py"

I don't advice you to set this attribut: maxHotSpanSecs
By defaults it's set to 7776000 seconds (90 days).
NOTE: If you set this too small, you can get an explosion of hot/warm

0 Karma

martin_mueller
SplunkTrust
SplunkTrust

The key settings for short retention times are these two indexes.conf:

frozenTimePeriodInSecs = <nonnegative integer>
maxHotSpanSecs = <positive integer>

The first setting defaults to six years, you'll want to set that to two months so 5270400 (61 days) or however long your months are in seconds. The second setting defaults to 90 days, which is great for six years of retention but not great for two months. A bucket is removed when its youngest event crosses the frozenTimePeriodInSecs, so you would effectively wait five months. Set this to a week or so to actually remove your data close to those two months.

See http://docs.splunk.com/Documentation/Splunk/6.2.2/admin/indexesconf for reference.

martin_mueller
SplunkTrust
SplunkTrust

That's an internal space Splunk uses to store block signatures. Normally you do not change the configuration for it.

0 Karma

martin_mueller
SplunkTrust
SplunkTrust

Every settings exists once for every index, the one under [_internal] applies to the _internal index while the one under [default] applies to all indexes that don't set their own value. You should modify the settings for the indexes you want to modify.

0 Karma

sbenamro
New Member

thanks for your help ! much appreciated.
one last question - what is the _blocksignature ?

0 Karma

sbenamro
New Member

Thanks for your help.
I'm using only the file over the local folder.
yet - I've noticed that I have "maxHotSpanSecs " in several places in that file, which are [default] and [_internal]

and I've noticed that "frozenTimePeriodInSecs " exist in several places as well which are - [default] and [_blocksignature] and [_internal] and [_introspection] and [_thefishbucket] and [history]

so where should I change it ?

0 Karma

martin_mueller
SplunkTrust
SplunkTrust

You should set this for the index you want to modify in the local directory of that index' app, or in system/local if the index is defined in system. Do not modify .conf files in any default directory.
Which index are you trying to modify?

0 Karma

sbenamro
New Member

thanks for the answer but under which section should I change it ?
I have for example - [_internal] or [default]

0 Karma
Get Updates on the Splunk Community!

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...