Splunk Search

Using maxHotSpanSecs of 1 hour or 1 day generated too many buckets

yannK
Splunk Employee
Splunk Employee

Following this advice, I decided to rotate my hot buckets every hour. (each bucket should contains only 1 hour of data, without overlapping 2 hours)

But I discovered that at the hour many buckets are being created. I have 3 hot buckets and they close and new open all the time.
This seems to be because I have some servers with a time drift, so I was receiving events from different hours, over more than 3 hours (maxHotBuckets=3)

maxHotSpanSecs =
* Upper bound of timespan of hot/warm buckets in seconds.
* Defaults to 7776000 seconds (90 days).
* NOTE: If you set this too small, you can get an explosion of hot/warm
buckets in the filesystem.
* If you set this parameter to less than 3600, it will be automatically reset to
3600, which will then activate snapping behavior (see below).
* This is an advanced parameter that should be set
with care and understanding of the characteristics of your data.
* If set to 3600 (1 hour), or 86400 (1 day), becomes also the lower bound
of hot bucket timespans. Further, snapping behavior (i.e. ohSnap)
is activated, whereby hot bucket boundaries will be set at exactly the hour
or day mark, relative to local midnight.
* Highest legal value is 4294967295

Tags (2)
1 Solution

yannK
Splunk Employee
Splunk Employee

My workaround was to use values for maxHotSpanSecs different from 3600 (1 hour), or 86400 (1 day).

Instead I used one hourless 1 second, or one day less 1 second.
Then my bucket span became a flexible rolling window, instead of a fixed window.

example for ~ 24h buckets
maxHotSpanSecs=86399

View solution in original post

bbeard
Engager

Your example for 24h (86399) looks correct. If you set to 3599 though (one hour less one second) you will invoke: If you set this parameter to less than 3600, it will be automatically reset to 3600, which will then activate snapping behavior (see below). You want to use at least 3601.

0 Karma

yannK
Splunk Employee
Splunk Employee

My workaround was to use values for maxHotSpanSecs different from 3600 (1 hour), or 86400 (1 day).

Instead I used one hourless 1 second, or one day less 1 second.
Then my bucket span became a flexible rolling window, instead of a fixed window.

example for ~ 24h buckets
maxHotSpanSecs=86399

Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...