Deployment Architecture

Why does the newest warm bucket roll to cold instead of the oldest warm bucket?

Chrisdarn
New Member

This is my indexes.conf file:

# volume definitions
[volume:hot]
path = /data/hot
maxVolumeDataSizeMB = 8500
[volume:cold]
path = /data/cold
maxVolumeDataSizeMB = 10500
# index definition (calculation is based on a single index)
[myindex]
homePath = volume:hot/myindex/db
coldPath = volume:cold/myindex/colddb
thawedPath = $SPLUNK_DB/myindex/thaweddb
coldToFrozenDir = /data/cold/myindex/frozendb
maxDataSize = 20
maxHotBuckets = 2
maxWarmDBCount = 5

When I upload data to Splunk Web, it creates a hot bucket. When I upload more data, this hot bucket fills and when it reaches 20mb it rolls to a warm bucket because of maxDataSize = 20. When there are 6 warm buckets, the 6th warm bucket rolls to cold. However my problem is that it appears the newest warm bucket moves to cold instead of the oldest. Could anyone explain why this is happening?

Example:
I have warm buckets db_1, db_2, db_3, db_4, db_5
when the newest db_6 is created, it moves straight to cold when it should move db_1 instead

0 Karma

gjanders
SplunkTrust
SplunkTrust

A maxDataSize of 20 seems far too small, Splunk defaults to 750MB.

From my own notes on this:
"When an index hits the size limit it will roll buckets to frozen based on the oldest bucket, even though this bucket may contain both old and new data"

In other words, the oldest date of the bucket is the one chosen to roll to cold, perhaps the bucket that has the "newest" data also has the oldest data?

This same answer might relate to your frozen data question as well, the buckets will roll to frozen based on the oldest datestamp when the size limits are hit.
Without the size limits the bucket would roll to frozen when the newest date is past the frozen time period...

0 Karma
Get Updates on the Splunk Community!

Built-in Service Level Objectives Management to Bridge the Gap Between Service & ...

Wednesday, May 29, 2024  |  11AM PST / 2PM ESTRegister now and join us to learn more about how you can ...

Get Your Exclusive Splunk Certified Cybersecurity Defense Engineer Certification at ...

We’re excited to announce a new Splunk certification exam being released at .conf24! If you’re headed to Vegas ...

Share Your Ideas & Meet the Lantern team at .Conf! Plus All of This Month’s New ...

Splunk Lantern is Splunk’s customer success center that provides advice from Splunk experts on valuable data ...