Why the function "strftime" not working in my search query?

[chrismok]

alt textIf I use this, no event return

sourcetype=abc source="*"+strftime(now(),"%Y%m%d")+"*"

But when I modify the query to

sourcetype=abc source="*20141104*"

There is a events return.

May I know is that a bug in Splunk?

[peter_krammer · Accepted solution]

Here is what you are looking for

sourcetype=abc [|stats count | eval source = "*"+strftime(now(),"%Y%m%d")+"*" | fields source | format]

Edited Answer to show the better performance solution found by davebrooking, but optimized a little by me.

[chrismok]

Tested for your solution, it can provide the better performance

[vasanthmss]

Hi chrismok,

Its not a bug in splunk,

strftime is a function that takes epoch time as first parameter and format human readable format like YYYYDDMM etc, based on your format string in second param.

you should use those functions in "eval".

As per your requirement this query will help you.

sourcetype=abc [|gentimes start=-1 | eval source="*"+strftime(now(),"%Y%m%d")+"*" | return source]

Thanks,
Vasu

[davebrooking]

What if you try adding the specific field name to the search - something like
sourcetype=abc source=[|gentimes start=-1 | eval source=""+strftime(now(),"%Y%m%d")+"" | return source]

Does that improve the performance?

[chrismok]

Error in 'search' command: Unable to parse the search: Comparator '=' has an invalid term on the left hand side.

[davebrooking]

Sorry, I was using the concept from this answer, but had no Splunk instance to test on.

I've since had a chance to try the technique on a small subset of data and noticed quite sizeable differences in execution time depending on whether you use stats count or gentimes start=-1 at the start of the subsearch with return or table as the last command in the subsearch. I'm getting consistently better execution times using

sourcetype=abc [|stats count | eval source="*"+strftime(now(),"%Y%m%d")+"*" | table source]

[chrismok]

Thanks vasanthmss. But I found that the performance will be decreased using your approach

sourcetype=abc [|gentimes start=-1 | eval source="*"+strftime(now(),"%Y%m%d")+"*" | return source] 

Run time: 3~4 seconds, Retrieve events: 18824

sourcetype=abc source="*20141104*"

Run time: 1 seconds, Retrieve events: 18824

[MuS]

Hi chrismok,

you're absolutly right regarding the performance. Try this instead:

... | eval file_date=strftime(now(), "%Y%m%d") | eval mySource="*" + file_date + "*" |  where source=mySource | ...

hope this helps ...

cheers, MuS

[chrismok]

Hi MuS.

Using your script, no record found.....

Regards,
Chris

[MuS]

upps my bad...try this updated command:

... | eval file_date=strftime(now(), "%Y%m%d") | eval mySource="." + file_date + "." | where match(source, mySource) | ...

[chrismok]

worser

[MuS]

another approach just came up my mind:
if you always need today's or yesterday's date in the source name, than you could use an eval based macro containing something like this:

strftime(relative_time(time(), "-d"), "%Y%m%d") 

If your macro is named yesterday you can use it like this in your searches:

 sourcetype=depmon_sys_rel_log  source=*`yesterday`* | ...

[chrismok]

It's not worked... as the Splunk macro is not similar to excel macro or function.... The Splunk only copy the macro string and place to the query

for example.

sourcetype=depmon_sys_rel_log source=`get_today`

Macro: get_today

strftime(now(),"%Y%m%d")

After ran the query and click the Search job inspecor. You can see that

search sourcetype=depmon_sys_rel_log source=strftime(now(),"%Y%m%d")

As a result, it is not worked

[MuS]

Well there must be something wrong, because this works for sure. If I run a search like this:

index=main source=*`yesterday`* 

it becomes this litsearch

litsearch index=main source=*04-Nov-2014*

and returns events ....

[chrismok]

Please take a look

[MuS]

chrismok, do you have any results on this by now? I wonder which approach will be the fastest for you. Thanks for sharing the results.

[chrismok]

Thank MuS. Just tried these 2 methods over 50 time. the process time are the same most of the time. But sometime your approach will faster around 0.02 ~ 0.06 seconds (Elapse time is 4 seconds). But the one bad thing is the star symbol cannot combine into the marco

[MuS]

And why could this be bad?

[chrismok]

Hi Mus,

Will check it next Monday as I am working in other project now..... Sorry.....

Regards,
Chris

[MuS]

Make sure you tick the Use eval-based definition? in the macro settings! Then it will work and should be pretty fast as well 😉

hmm, cannot add a screenshot here???