I am ingesting data at 6AM, 2PM, 7PM, 10PM (CST)
Is there anyway I could have my query check the time and set earliest to the time that it past last?
ie:
It is 8AM CST time, the search would set earliest to 6AM
It is 9PM CST time, the search would set earliest to 7PM
Thanks
I thought about it a little more and something like this should work. It is certainly not perfect though - you would want to check month and year crossovers also to guarantee accuracy. Change log_level to WARN
or INFO
if you're environment is perfect and doesn't have errors.
index=_internal log_level=ERROR
| eval curr_date_hour = strftime(now(), "%H"),
curr_date_mday = strftime(now(), "%d")
| eval search_mday = case(curr_date_hour <= 6, curr_date_mday - 1,
curr_date_hour > 6, curr_date_mday)
| eval search_hour = case(curr_date_hour < 6, 22,
curr_date_hour > 22, 22,
curr_date_hour > 19, 19,
curr_date_hour > 14, 14,
curr_date_hour > 6, 6)
| where (search_hour != 22 AND search_mday = date_mday AND date_hour >= search_hour) OR
(search_hour = 22 AND search_mday = date_mday AND date_hour >= search_hour) OR
(search_hour = 22 AND search_mday + 1 = date_mday)
I thought about it a little more and something like this should work. It is certainly not perfect though - you would want to check month and year crossovers also to guarantee accuracy. Change log_level to WARN
or INFO
if you're environment is perfect and doesn't have errors.
index=_internal log_level=ERROR
| eval curr_date_hour = strftime(now(), "%H"),
curr_date_mday = strftime(now(), "%d")
| eval search_mday = case(curr_date_hour <= 6, curr_date_mday - 1,
curr_date_hour > 6, curr_date_mday)
| eval search_hour = case(curr_date_hour < 6, 22,
curr_date_hour > 22, 22,
curr_date_hour > 19, 19,
curr_date_hour > 14, 14,
curr_date_hour > 6, 6)
| where (search_hour != 22 AND search_mday = date_mday AND date_hour >= search_hour) OR
(search_hour = 22 AND search_mday = date_mday AND date_hour >= search_hour) OR
(search_hour = 22 AND search_mday + 1 = date_mday)
try earliest=-2h
in your search query?
I'm sure it's possible, but the easier way to do it would be to:
earliest=-6h
, ordedup
to ensure your results only contain the latest data.