I have one thread of data that we'd like to keep basically forever. Over the past 8 years the log has only grown to 210MB so we definitely do not have a problem with size or space.
I Splunk'd this log into it's own index, not mixed in with any other indexes so I could keep the retention settings different. The index settings are (including the defaults):
system assureUTF8 = false
system blockSignSize = 0
system blockSignatureDatabase = _blocksignature
system bucketRebuildMemoryHint = auto
system coldPath = volume:cold1/gud
system coldPath.maxDataSizeMB = 150000
system coldToFrozenDir =
system coldToFrozenScript =
system compressRawdata = true
system defaultDatabase = main
system enableOnlineBucketRepair = true
system enableRealtimeSearch = true
system frozenTimePeriodInSecs = 188697600
system homePath = volume:hot1/gud
system homePath.maxDataSizeMB = 50000
system indexThreads = auto
system maxBloomBackfillBucketAge = 30d
system maxConcurrentOptimizes = 3
system maxDataSize = auto
system maxHotBuckets = 3
system maxHotIdleSecs = 0
system maxHotSpanSecs = 7776000
system maxMemMB = 5
system maxMetaEntries = 1000000
system maxRunningProcessGroups = 20
system maxRunningProcessGroupsLowPriority = 1
system maxTotalDataSizeMB = 500000
system maxWarmDBCount = 300
system memPoolMB = auto
system minRawFileSyncSecs = disable
system partialServiceMetaPeriod = 0
system quarantineFutureSecs = 2592000
system quarantinePastSecs = 77760000
system rawChunkSizeBytes = 131072
system rotatePeriodInSecs = 60
system serviceMetaPeriod = 25
system suppressBannerList =
system sync = 0
system syncMeta = true
system thawedPath = $SPLUNK_COLDDB/gud/thaweddb
system throttleCheckPeriod = 15
The problem is this index is deleting data older than about 120 days. The total size of the index is 3MB right now.
According to the way I read this configuration it should not freeze data until it reaches 50GB (homePath.maxDataSizeMB = 50000) or approximately 5.98 years old (frozenTimePeriodInSecs = 188697600).
So why then is it deleting data from the index so soon?
because a bucket roll to frozen when :
This is very common if your buckets are new and small.
FYI a hot bucket roll when it reaches : maxHotSpanSecs, maxHotBuckets, maxDataSize (that depends of the system, and ca go tup to 10GB per bucket)
use the| dbinspect index=myindex
to check the state of your buckets.