Hi,
As you know, date_month, date_mday, date_year fields are so useful to fetch data quickly.
I usually use these fields in my main search, but since the value of the date_month field also contains alphanumeric (a-z) characters, not just numbers, I always face problems using it in a search.
Sure, there are a lot of ways to convert it to numeric such as using lookup table or strftime eval function, but all of these ways lose performance to fetch data at least 2~4 times.
I really want to know a way to use numeric date_month field with minimize to lose data fetch performance.
You may not realize this, but although the date*
fields are a convenient by-product of timestamping, they are created BEFORE the TZ is applied and are therefore usually off by hours. Rare is it that any system timestamps in GMT so the only time your date_month
field is "correct" is when date_zone=0
or date_zone=local
(and most of the time, when date_zone=local
the timestamp itself is wrong because somebody forgot to put in a TZ=
into that props.conf
). For this reason, I rarely use them and create my own like this:
| eval month=strftime(_time,"%m")
For more info, see here:
http://answers.splunk.com/answers/60143/is-there-a-numeric-alternative-to-date-month.html
You may not realize this, but although the date*
fields are a convenient by-product of timestamping, they are created BEFORE the TZ is applied and are therefore usually off by hours. Rare is it that any system timestamps in GMT so the only time your date_month
field is "correct" is when date_zone=0
or date_zone=local
(and most of the time, when date_zone=local
the timestamp itself is wrong because somebody forgot to put in a TZ=
into that props.conf
). For this reason, I rarely use them and create my own like this:
| eval month=strftime(_time,"%m")
For more info, see here:
http://answers.splunk.com/answers/60143/is-there-a-numeric-alternative-to-date-month.html
I think using eval function has a little bit loss of fetching data performance.
I'll compare the ways between using eval function and using props.conf.
Thanks all.
You can use the job inspector
to evaluate and compare.
You can save this "Month" field as calculated field in your props.conf and get better performance.