Deployment Architecture

Why does the newest warm bucket roll to cold instead of the oldest warm bucket?

Chrisdarn
New Member

This is my indexes.conf file:

# volume definitions
[volume:hot]
path = /data/hot
maxVolumeDataSizeMB = 8500
[volume:cold]
path = /data/cold
maxVolumeDataSizeMB = 10500
# index definition (calculation is based on a single index)
[myindex]
homePath = volume:hot/myindex/db
coldPath = volume:cold/myindex/colddb
thawedPath = $SPLUNK_DB/myindex/thaweddb
coldToFrozenDir = /data/cold/myindex/frozendb
maxDataSize = 20
maxHotBuckets = 2
maxWarmDBCount = 5

When I upload data to Splunk Web, it creates a hot bucket. When I upload more data, this hot bucket fills and when it reaches 20mb it rolls to a warm bucket because of maxDataSize = 20. When there are 6 warm buckets, the 6th warm bucket rolls to cold. However my problem is that it appears the newest warm bucket moves to cold instead of the oldest. Could anyone explain why this is happening?

Example:
I have warm buckets db_1, db_2, db_3, db_4, db_5
when the newest db_6 is created, it moves straight to cold when it should move db_1 instead

0 Karma

gjanders
SplunkTrust
SplunkTrust

A maxDataSize of 20 seems far too small, Splunk defaults to 750MB.

From my own notes on this:
"When an index hits the size limit it will roll buckets to frozen based on the oldest bucket, even though this bucket may contain both old and new data"

In other words, the oldest date of the bucket is the one chosen to roll to cold, perhaps the bucket that has the "newest" data also has the oldest data?

This same answer might relate to your frozen data question as well, the buckets will roll to frozen based on the oldest datestamp when the size limits are hit.
Without the size limits the bucket would roll to frozen when the newest date is past the frozen time period...

0 Karma
Get Updates on the Splunk Community!

Webinar Recap | Revolutionizing IT Operations: The Transformative Power of AI and ML ...

The Transformative Power of AI and ML in Enhancing Observability   In the realm of IT operations, the ...

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

ICYMI - Check out the latest releases of Splunk Edge Processor

Splunk is pleased to announce the latest enhancements to Splunk Edge Processor.  HEC Receiver authorization ...