Splunk Search

Is there any way to use the date_month field in a search as a numeric field without performance loss?

Riel
Engager

Hi,

As you know, date_month, date_mday, date_year fields are so useful to fetch data quickly.
I usually use these fields in my main search, but since the value of the date_month field also contains alphanumeric (a-z) characters, not just numbers, I always face problems using it in a search.
Sure, there are a lot of ways to convert it to numeric such as using lookup table or strftime eval function, but all of these ways lose performance to fetch data at least 2~4 times.

I really want to know a way to use numeric date_month field with minimize to lose data fetch performance.

1 Solution

woodcock
Esteemed Legend

You may not realize this, but although the date* fields are a convenient by-product of timestamping, they are created BEFORE the TZ is applied and are therefore usually off by hours. Rare is it that any system timestamps in GMT so the only time your date_month field is "correct" is when date_zone=0 or date_zone=local (and most of the time, when date_zone=local the timestamp itself is wrong because somebody forgot to put in a TZ= into that props.conf). For this reason, I rarely use them and create my own like this:

| eval month=strftime(_time,"%m")

For more info, see here:

http://answers.splunk.com/answers/60143/is-there-a-numeric-alternative-to-date-month.html

View solution in original post

woodcock
Esteemed Legend

You may not realize this, but although the date* fields are a convenient by-product of timestamping, they are created BEFORE the TZ is applied and are therefore usually off by hours. Rare is it that any system timestamps in GMT so the only time your date_month field is "correct" is when date_zone=0 or date_zone=local (and most of the time, when date_zone=local the timestamp itself is wrong because somebody forgot to put in a TZ= into that props.conf). For this reason, I rarely use them and create my own like this:

| eval month=strftime(_time,"%m")

For more info, see here:

http://answers.splunk.com/answers/60143/is-there-a-numeric-alternative-to-date-month.html

Riel
Engager

I think using eval function has a little bit loss of fetching data performance.
I'll compare the ways between using eval function and using props.conf.
Thanks all.

0 Karma

woodcock
Esteemed Legend

You can use the job inspector to evaluate and compare.

0 Karma

somesoni2
SplunkTrust
SplunkTrust

You can save this "Month" field as calculated field in your props.conf and get better performance.

Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...