Getting Data In

date_month localization and reporting for the previous month

decoherence
Explorer

Hello! Splunk n00b looking for confirmation of something! I can't find documentation for date_month that specifies whether it is localized. It appears that it isn't. Given this example search, which spans Feb 28 and Mar 1 in my timezone:

... earliest=1551384000 latest=1551416401
| eval date_month_local=strftime(_time, "%B")
| search date_month=march date_month_local=February

Which returns results claiming to be in both March and February starting at 7pm local (I'm UTC-0500). That makes sense if date_month is based on UTC.*

Given that, if I have a report that I want to run on the previous month (Feb, in this example,) and I want a field with the name of the month in that report, should I be using strftime(_time, "%B") instead of date_month? If I use date_month, the report (which runs in -0500) ends up with a 'march' value for records after Feb 28 7pm -0500.

If I haven't completely missed something then all this might seem a bit self evident (and hopefully helpful for someone.) There was some surprise amongst management when it seemed that date_month was not localized. That's why I want to be very sure I have this right. If anyone knows documentation that describes or implies how date_month treats timezones, I would appreciate being pointed at it.

Thanks for your time!

*UPDATE: As described by somesoni2 in the accepted answer, date_month is based on the literal string at index time. In this case, the indexer was looking at a timestamp field which happened to be in UTC.

0 Karma
1 Solution

somesoni2
SplunkTrust
SplunkTrust

The date_* fields are generated at index time and datetime values are the literal values parsed from the event when it is indexed, regardless of its timezone. So, the data might be generated, parsed and indexed on UTC time (check your indexer/heavy forwarder time zone) and then is translated to your timezone while searching. (See this for more information https://docs.splunk.com/Documentation/Splunk/7.2.4/Knowledge/Usedefaultfields#Default_datetime_field...)

The best way here would be to use _time based month calculation as that would honor your timezone/time range.

View solution in original post

somesoni2
SplunkTrust
SplunkTrust

The date_* fields are generated at index time and datetime values are the literal values parsed from the event when it is indexed, regardless of its timezone. So, the data might be generated, parsed and indexed on UTC time (check your indexer/heavy forwarder time zone) and then is translated to your timezone while searching. (See this for more information https://docs.splunk.com/Documentation/Splunk/7.2.4/Knowledge/Usedefaultfields#Default_datetime_field...)

The best way here would be to use _time based month calculation as that would honor your timezone/time range.

decoherence
Explorer

Just to clarify my own understanding, my assumption that date_month is always based on UTC is not right? Instead, the value of date_month is dependent on the indexer/heavy forwarder (which in this case appears to be set to UTC)?

Thanks again. I will look in to how to confirm the timezone configuration of our system.

0 Karma

somesoni2
SplunkTrust
SplunkTrust

The date_* are parsed based on the literal value of the time fields, regardless of the time zone. So for date string 05:22:21 will be parsed into indexed fields: date_hour::5 date_minute::22 date_second::21.

0 Karma

decoherence
Explorer

I'm not sure I understand what you mean by 'literal value of the time fields.' Are you talking about unix epoch time?

0 Karma

somesoni2
SplunkTrust
SplunkTrust

So if your raw data has timestamp as 2019-03-29 21:20:20 CST OR 2019-03-29 21:20:20 UTC, they both will have same date_* field values. It's always going to take 21 as date_hour, doesn't matter what the timezone is.

0 Karma

decoherence
Explorer

Right on, I follow you now. Cheers 🙂

0 Karma
Get Updates on the Splunk Community!

Welcome to the Splunk Community!

(view in My Videos) We're so glad you're here! The Splunk Community is place to connect, learn, give back, and ...

Tech Talk | Elevating Digital Service Excellence: The Synergy of Splunk RUM & APM

Elevating Digital Service Excellence: The Synergy of Real User Monitoring and Application Performance ...

Adoption of RUM and APM at Splunk

    Unleash the power of Splunk Observability   Watch Now In this can't miss Tech Talk! The Splunk Growth ...