Hello Experts,
Is there a way to get the current policies that are defined for backups?
How often/when does the index data move from hot db to warm db
How often/when does the index data move from warm db to cold db
How often/when does the index data move from cold db to frozen and removed all together
We are trying to make sure index data is archived indefinitely and not removed at all due to compliance purposes.
Thanks,
Yes you can get the current policies that are defined for backups by reading your indexes.conf configuration file
For more information, Read here: http://docs.splunk.com/Documentation/Splunk/6.2.2/Indexer/Setaretirementandarchivingpolicy
For other information concerning hot, warm ....buckets, start here http://docs.splunk.com/Documentation/Splunk/6.2.2/Indexer/Backupindexeddata
There is great app to monitor Splunk's Health. See this https://splunkbase.splunk.com/app/1919/
You've a dashboard "Available Indexes" in this app which can provide you list of indexes with its current Retention period.
This dashboard is based on the result of Splunk REST Api endpoint for Indexes. Use following query to get more detailed information about the your Splunk indexes.
| rest /services/data/indexes
Many thanks to the link to the app and the command.
Also executing btool on the indexes configuration with the --debug flag will show which indexes.conf file is used in setting these retention attributes:
./Splunk btool indexes list --debug
This can be redirected to a txt file for additional analysis in a text editor like vim.
Thanks. This is interesting data.
Yes you can get the current policies that are defined for backups by reading your indexes.conf configuration file
For more information, Read here: http://docs.splunk.com/Documentation/Splunk/6.2.2/Indexer/Setaretirementandarchivingpolicy
For other information concerning hot, warm ....buckets, start here http://docs.splunk.com/Documentation/Splunk/6.2.2/Indexer/Backupindexeddata
Thanks for these links,
I got the following configured,
My maxDataSize = auto
What is the auto setting?
maxWarmDBCount = 300 ; so that means I can have 300 warm buckets before it is moved to frozen by default
My maxTotalDataSizeMB = 500000 ; I'm assuming this is not equal to maxDataSize
1- Here is the syntaxe for maxDataSize :
maxDataSize = |auto|auto_high_volume
2- Here is the syntaxe for maxWarmDBCount:
maxWarmDBCount =<nonnegative integer>
This is the maximum number of warm buckets.
- Warm buckets are located in the for the index.
- If set to zero, it will not retain any warm buckets (will roll them to cold as soon as it can)
- Defaults to 300.
- Highest legal value is 4294967295
3-Here is the syntaxe for maxTotalDataSizeMB :
maxTotalDataSizeMB = <nonnegative integer>
For more informations, start reading here: http://docs.splunk.com/Documentation/Splunk/6.2.2/Admin/Indexesconf
Thanks for the detailed response. The links are of super help.