Deployment Architecture

Why does the newest warm bucket roll to cold instead of the oldest warm bucket?

Chrisdarn
New Member

This is my indexes.conf file:

# volume definitions
[volume:hot]
path = /data/hot
maxVolumeDataSizeMB = 8500
[volume:cold]
path = /data/cold
maxVolumeDataSizeMB = 10500
# index definition (calculation is based on a single index)
[myindex]
homePath = volume:hot/myindex/db
coldPath = volume:cold/myindex/colddb
thawedPath = $SPLUNK_DB/myindex/thaweddb
coldToFrozenDir = /data/cold/myindex/frozendb
maxDataSize = 20
maxHotBuckets = 2
maxWarmDBCount = 5

When I upload data to Splunk Web, it creates a hot bucket. When I upload more data, this hot bucket fills and when it reaches 20mb it rolls to a warm bucket because of maxDataSize = 20. When there are 6 warm buckets, the 6th warm bucket rolls to cold. However my problem is that it appears the newest warm bucket moves to cold instead of the oldest. Could anyone explain why this is happening?

Example:
I have warm buckets db_1, db_2, db_3, db_4, db_5
when the newest db_6 is created, it moves straight to cold when it should move db_1 instead

0 Karma

gjanders
SplunkTrust
SplunkTrust

A maxDataSize of 20 seems far too small, Splunk defaults to 750MB.

From my own notes on this:
"When an index hits the size limit it will roll buckets to frozen based on the oldest bucket, even though this bucket may contain both old and new data"

In other words, the oldest date of the bucket is the one chosen to roll to cold, perhaps the bucket that has the "newest" data also has the oldest data?

This same answer might relate to your frozen data question as well, the buckets will roll to frozen based on the oldest datestamp when the size limits are hit.
Without the size limits the bucket would roll to frozen when the newest date is past the frozen time period...

0 Karma
Get Updates on the Splunk Community!

Introducing the 2024 SplunkTrust!

Hello, Splunk Community! We are beyond thrilled to announce our newest group of SplunkTrust members!  The ...

Introducing the 2024 Splunk MVPs!

We are excited to announce the 2024 cohort of the Splunk MVP program. Splunk MVPs are passionate members of ...

Splunk Custom Visualizations App End of Life

The Splunk Custom Visualizations apps End of Life for SimpleXML will reach end of support on Dec 21, 2024, ...