Hi yannk, thanks for the answer - very useful and I think I have a handle on it now in terms of the various states. I just have one more question - if we were to set maxTotalDataSizeMB and frozenTimePeriodInSecs to ensure that we don't keep too much data in active indexes, but due to the default settings within indexes.conf meant that data had not moved from WARM to COLD would that data be moved from WARM to FROZEN directly?
the lifetime of a bucket is :
So a bucket can go directly from warm to frozen without being cold.
The Cold state is optional, you may end up with an empty coldPath with the splunk defaults. And is only useful if you are using homePath and coldPath on different partitions for space reasons.
Read indexes.conf specifications. You will need to edit a config file.
see http://docs.splunk.com/Documentation/Splunk/6.2.3/Indexer/Setaretirementandarchivingpolicy
setup a ** frozenTimePeriodInSecs** to be the day limit. (default 6 years)
and if you retention is very low (less than months) , you may also want to reduce the maxHotSpanSecs (maximum hot bucket timespan default to 90 days) to force them to rotate more often. (as hot buckets cannot be frozen)
Remark : do not use exactly 1 hour or 1 day for maxHotSpanSecs, It's best to use one week, it prevent too many buckets to be created if your events are not received chronologically.