Is there a way to get the current retention and archiving policies that are defined for backups?

[kkossery]

Hello Experts,

Is there a way to get the current policies that are defined for backups?
How often/when does the index data move from hot db to warm db
How often/when does the index data move from warm db to cold db
How often/when does the index data move from cold db to frozen and removed all together

We are trying to make sure index data is archived indefinitely and not removed at all due to compliance purposes.

Thanks,

[somesoni2]

There is great app to monitor Splunk's Health. See this https://splunkbase.splunk.com/app/1919/

You've a dashboard "Available Indexes" in this app which can provide you list of indexes with its current Retention period.

This dashboard is based on the result of Splunk REST Api endpoint for Indexes. Use following query to get more detailed information about the your Splunk indexes.

| rest /services/data/indexes

[kkossery]

Many thanks to the link to the app and the command.

[transtrophe]

Also executing btool on the indexes configuration with the --debug flag will show which indexes.conf file is used in setting these retention attributes:

./Splunk btool indexes list --debug

This can be redirected to a txt file for additional analysis in a text editor like vim.

[kkossery]

Thanks. This is interesting data.

[stephanefotso · Accepted solution]

Yes you can get the current policies that are defined for backups by reading your indexes.conf configuration file
For more information, Read here: http://docs.splunk.com/Documentation/Splunk/6.2.2/Indexer/Setaretirementandarchivingpolicy

For other information concerning hot, warm ....buckets, start here http://docs.splunk.com/Documentation/Splunk/6.2.2/Indexer/Backupindexeddata

[kkossery]

Thanks for these links,
I got the following configured,
My maxDataSize = auto
What is the auto setting?
maxWarmDBCount = 300 ; so that means I can have 300 warm buckets before it is moved to frozen by default

My maxTotalDataSizeMB = 500000 ; I'm assuming this is not equal to maxDataSize

[stephanefotso]

1- Here is the syntaxe for maxDataSize :

maxDataSize = |auto|auto_high_volume

• This is The maximum size in MB for a hot DB to reach before a roll to warm is triggered.
• Specifying "auto" or "auto_high_volume" will cause Splunk to autotune this parameter (recommended).
• You should use "auto_high_volume" for high-volume indexes (such as the main index); otherwise, use "auto". A "high volume index" would typically be considered one that gets over 10GB of data per day.
• Defaults to "auto", which sets the size to 750MB.

2- Here is the syntaxe for maxWarmDBCount:

maxWarmDBCount =<nonnegative integer>

This is the maximum number of warm buckets.
- Warm buckets are located in the for the index.
- If set to zero, it will not retain any warm buckets (will roll them to cold as soon as it can)
- Defaults to 300.
- Highest legal value is 4294967295

3-Here is the syntaxe for maxTotalDataSizeMB :

maxTotalDataSizeMB = <nonnegative integer>

• This is the maximum size of an index (in MB).
• If an index grows larger than the maximum size, the oldest data is frozen.
• This parameter only applies to hot, warm, and cold buckets. It does not apply to thawed buckets.
• Defaults to 500000.
• Highest legal value is 4294967295

For more informations, start reading here: http://docs.splunk.com/Documentation/Splunk/6.2.2/Admin/Indexesconf

[kkossery]

Thanks for the detailed response. The links are of super help.