Deployment Architecture

My warm bucket moves straight to frozen and I don't know why?

Chrisdarn
New Member

This is my indexes.conf:
*[volume:hot]
path = /data/hot
maxVolumeDataSizeMB = 8500
[volume:cold]
path = /data/cold
maxVolumeDataSizeMB = 10500
[myindex]
homePath = volume:hot/myindex/db
coldPath = volume:cold/myindex/colddb
thawedPath = /data/cold/myindex/thaweddb
coldToFrozenDir = /data/cold/myindex/frozendb

hot to warm

maxDataSize = 20

2 hours to move from hot to warm

maxHotSpanSecs = 7200

100mb for cold to fill up

coldPath.maxDataSizeMB = 100
maxTotalDataSizeMB = 150

maximum warm buckets

maxWarmDBCount = 2
frozenTimePeriodInSecs = 1800
*

I uploaded a 2mb zip which extracts to 18mb of files onto the webUI. It creates a hot bucket in the hot/warm db folder. I upload the same zip again onto the webUI which then converts the hot bucket to a warm bucket due to maxDataSize = 20 with the remainder going into a new hot bucket. After a minute or so the warm bucket moves directly to frozen without going to cold. Also, maxHotSpanSecs = 7200 doesn't seem to work either ie my hot buckets don't roll to warm after 7200 seconds (2 hours) Could anyone tell me what is wrong with my indexes.conf file?
Thank you!

0 Karma

kellewic
Path Finder

To add to teunlaan's answer - if all events in your bucket are older than 1800 seconds it will freeze. You can check buckets using this search:

|dbinspect index=YOUR_INDEX_NAME
|eval span=replace(replace(replace(replace(tostring(ceiling(endEpoch-startEpoch), "duration"),"(?:(\d+)\+?)(\d+)\:(\d+):(\d+)","\1d \2h \3m \4s"), "(?<!\d)0+[dhms]", ""), "^\s+$", "00s"), "0(?=\d[dhms])", "")
|convert ctime(startEpoch) as startDate, ctime(endEpoch) as endDate
|table splunk_server, index, state, startDate, endDate, span, sizeOnDiskMB

To answer your other question about maxHotSpanSecs not working, it's because you hit the maxDataSize first so the hot rolls to warm based on size. The first condition to hit (either time or size) wins.

0 Karma

teunlaan
Contributor

frozenTimePeriodInSecs = 1800

What is the Timestamp of the logs?

If they are older than 1800 seconds, (if they alre the only event in the bucket)

0 Karma
Get Updates on the Splunk Community!

Stay Connected: Your Guide to May Tech Talks, Office Hours, and Webinars!

Take a look below to explore our upcoming Community Office Hours, Tech Talks, and Webinars this month. This ...

They're back! Join the SplunkTrust and MVP at .conf24

With our highly anticipated annual conference, .conf, comes the fez-wearers you can trust! The SplunkTrust, as ...

Enterprise Security Content Update (ESCU) | New Releases

Last month, the Splunk Threat Research Team had two releases of new security content via the Enterprise ...