Splunk Search

Bucket command time boundary issues

gcoles
Communicator

I am writing a search that looks at weighted moving averages of data points summarized and logged at 2 minute intervals. I need to bucket the data into two minute spans, in a window of ten minutes. The search will run every minute, and look at the past ten minutes worth of data, thus, there should always be five buckets of 2 minutes each. You'd think this would be as easy as:

earliest=-10m@m latest=@m *base_search* | bucket _time span=2m | stats xxx by _time

However, the bucket command (and timechart, etc) always make bucket boundaries snap to even numbered time boundaries, rather than being relative to the search time boundaries. To elaborate, if the search is made at 10:10:23, there are five buckets, for 10:00, 10:02, 10:04, 10:06, and 10:00, and if the search is run at 10:11:xx, there are six buckets: 10:00, 10:02 ... 10:10, with the first and last bucket containing one minutes' worth of data each (half the data).

What I think should happen with the 10:11 search is five buckets, the first being 10:01, then 10:03, etc. Has anyone found a way to do this that still lets them sleep at night? Maybe a call to eval that segments time similarly to the bucket command (could be a macro).

1 Solution

dart
Splunk Employee
Splunk Employee

You could indeed work around this using eval:
earliest=-10m@m latest=@m *base_search* | addinfo | eval min_time = info_min_time | bucket span=2m info_min_time | eval info_min_time = strftime(info_min_time, "%H:%M:%S")| eval offset = min_time - info_min_time | eval _time=_time-offset | bucket span=2m _time | eval _time=_time+offset| stats xxx by _time

This is the search I used for testing my work:

sourcetype=access_combined earliest=-9m@m | addinfo | eval orig_time = strftime(_time, "%H:%M:%S")| eval min_time = info_min_time | bucket span=2m info_min_time | eval offset = min_time - info_min_time | eval _time=_time-offset| bucket span=2m _time | eval _time=_time+offset | eval min_time = strftime(min_time, "%H:%M:%S") | eval info_min_time = strftime(info_min_time, "%H:%M:%S")| table min_time info_min_time _time orig_time

View solution in original post

dart
Splunk Employee
Splunk Employee

You could indeed work around this using eval:
earliest=-10m@m latest=@m *base_search* | addinfo | eval min_time = info_min_time | bucket span=2m info_min_time | eval info_min_time = strftime(info_min_time, "%H:%M:%S")| eval offset = min_time - info_min_time | eval _time=_time-offset | bucket span=2m _time | eval _time=_time+offset| stats xxx by _time

This is the search I used for testing my work:

sourcetype=access_combined earliest=-9m@m | addinfo | eval orig_time = strftime(_time, "%H:%M:%S")| eval min_time = info_min_time | bucket span=2m info_min_time | eval offset = min_time - info_min_time | eval _time=_time-offset| bucket span=2m _time | eval _time=_time+offset | eval min_time = strftime(min_time, "%H:%M:%S") | eval info_min_time = strftime(info_min_time, "%H:%M:%S")| table min_time info_min_time _time orig_time

gcoles
Communicator

Thanks dart, this seems to work well for me. I'll try to make it into a macro so that the span time can be supplied as an argument and the macro used as a replacement for bucket.

0 Karma

gcoles
Communicator

Just filed an ER for it.

0 Karma

lguinn2
Legend

Regardless of the actual answer, please file an enhancement request / bug report at http://www.splunk.com/support - this behavior is not intuitive...

0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

Splunk is officially part of Cisco

Revolutionizing how our customers build resilience across their entire digital footprint.   Splunk ...

Splunk APM & RUM | Planned Maintenance March 26 - March 28, 2024

There will be planned maintenance for Splunk APM and RUM between March 26, 2024 and March 28, 2024 as ...