Deployment Architecture

My warm bucket moves straight to frozen and I don't know why?

Chrisdarn
New Member

This is my indexes.conf:
*[volume:hot]
path = /data/hot
maxVolumeDataSizeMB = 8500
[volume:cold]
path = /data/cold
maxVolumeDataSizeMB = 10500
[myindex]
homePath = volume:hot/myindex/db
coldPath = volume:cold/myindex/colddb
thawedPath = /data/cold/myindex/thaweddb
coldToFrozenDir = /data/cold/myindex/frozendb

hot to warm

maxDataSize = 20

2 hours to move from hot to warm

maxHotSpanSecs = 7200

100mb for cold to fill up

coldPath.maxDataSizeMB = 100
maxTotalDataSizeMB = 150

maximum warm buckets

maxWarmDBCount = 2
frozenTimePeriodInSecs = 1800
*

I uploaded a 2mb zip which extracts to 18mb of files onto the webUI. It creates a hot bucket in the hot/warm db folder. I upload the same zip again onto the webUI which then converts the hot bucket to a warm bucket due to maxDataSize = 20 with the remainder going into a new hot bucket. After a minute or so the warm bucket moves directly to frozen without going to cold. Also, maxHotSpanSecs = 7200 doesn't seem to work either ie my hot buckets don't roll to warm after 7200 seconds (2 hours) Could anyone tell me what is wrong with my indexes.conf file?
Thank you!

0 Karma

kellewic
Path Finder

To add to teunlaan's answer - if all events in your bucket are older than 1800 seconds it will freeze. You can check buckets using this search:

|dbinspect index=YOUR_INDEX_NAME
|eval span=replace(replace(replace(replace(tostring(ceiling(endEpoch-startEpoch), "duration"),"(?:(\d+)\+?)(\d+)\:(\d+):(\d+)","\1d \2h \3m \4s"), "(?<!\d)0+[dhms]", ""), "^\s+$", "00s"), "0(?=\d[dhms])", "")
|convert ctime(startEpoch) as startDate, ctime(endEpoch) as endDate
|table splunk_server, index, state, startDate, endDate, span, sizeOnDiskMB

To answer your other question about maxHotSpanSecs not working, it's because you hit the maxDataSize first so the hot rolls to warm based on size. The first condition to hit (either time or size) wins.

0 Karma

teunlaan
Contributor

frozenTimePeriodInSecs = 1800

What is the Timestamp of the logs?

If they are older than 1800 seconds, (if they alre the only event in the bucket)

0 Karma
Get Updates on the Splunk Community!

ICYMI - Check out the latest releases of Splunk Edge Processor

Splunk is pleased to announce the latest enhancements to Splunk Edge Processor.  HEC Receiver authorization ...

Introducing the 2024 SplunkTrust!

Hello, Splunk Community! We are beyond thrilled to announce our newest group of SplunkTrust members!  The ...

Introducing the 2024 Splunk MVPs!

We are excited to announce the 2024 cohort of the Splunk MVP program. Splunk MVPs are passionate members of ...