Splunk Search

Natural sort order instead of lexicographical order in search?

splunknoob408
Explorer

I've got a date field that I extracted from log messages, and it is pulled from two different sources. One source zero-pads the numbers, so I get dates like 12/08/17, while the other does not and gives me 12/8/17.

The problem is during sort, where I want a natural sort order, but the lexicographical sorting swaps the date entries around.

Is there a way in splunk to force it to zero-pad the dates so they are sorted properly? I have seen references to using a strptime() function, but don't know how to use it in my particular example. Here is my search, which works, except that the X axis (date) has its columns swapped around due to the lexicographical sorting:

source=OrderEventsTable service="tracking" event_details="shipped on*" | regex event_details="\d+/\d+/\d+" | dedup order_id | stats count as "shipped number" by shipped_date

I only know that I would want to pass strptime %m/%d/%y, since %d zero-pads the day. I tried this, unsuccessfully, because it changed shipped_date into a decimal number.

source=OrderEventsTable service="tracking" event_details="shipped on*" | regex event_details="\d+/\d+/\d+" | dedup order_id | stats count as "shipped number" by shipped_date | eval shipped_date=strptime(shipped_date,"%m/%d/%y")
0 Karma
1 Solution

MonkeyK
Builder

splunknoob408,
When sorting by time, epoch time (decimal number version) works best. You can convert to a human readable format after.

Try this:

source=OrderEventsTable service="tracking" event_details="shipped on*" 
| regex event_details="\d+/\d+/\d+" 
| dedup order_id 
| eval shipped_date=strptime(shipped_date,"%m/%d/%y") 
| stats count as "shipped number" by shipped_date 
| sort shipped_date
| eval shipped_date=strftime(shipped_date,"%m/%d/%y") 

View solution in original post

MonkeyK
Builder

splunknoob408,
When sorting by time, epoch time (decimal number version) works best. You can convert to a human readable format after.

Try this:

source=OrderEventsTable service="tracking" event_details="shipped on*" 
| regex event_details="\d+/\d+/\d+" 
| dedup order_id 
| eval shipped_date=strptime(shipped_date,"%m/%d/%y") 
| stats count as "shipped number" by shipped_date 
| sort shipped_date
| eval shipped_date=strftime(shipped_date,"%m/%d/%y") 

splunknoob408
Explorer

Awesome answer, thank you so much for the help! This totally worked.

0 Karma

niketn
Legend

@MonkeyK, after sorting epoch date, I would use fieldformat instead of the last eval. So that field remains epoch date and value converts to string date.

 | fieldformat shipped_date=strftime(shipped_date,"%m/%d/%y") 
____________________________________________
| makeresults | eval message= "Happy Splunking!!!"
0 Karma

MonkeyK
Builder

a new command! thank you! This will be useful in a number of my existing queries

0 Karma
Get Updates on the Splunk Community!

Stay Connected: Your Guide to May Tech Talks, Office Hours, and Webinars!

Take a look below to explore our upcoming Community Office Hours, Tech Talks, and Webinars this month. This ...

They're back! Join the SplunkTrust and MVP at .conf24

With our highly anticipated annual conference, .conf, comes the fez-wearers you can trust! The SplunkTrust, as ...

Enterprise Security Content Update (ESCU) | New Releases

Last month, the Splunk Threat Research Team had two releases of new security content via the Enterprise ...