Splunk Search

string to time and then timechart

KarunK
Contributor

Hi All,

I am trying to extract the timestamps from the log file name (source) and then find how many logs are produced at a span of 5 min - using "timechart span=5min". But i couldnt do it. It seems the timechart work only with field "_time". I have tried rename the log_time (extarcetd from the source) to _time.

| eval _time=log_time

But failed.............

I got it correct upto "| fields log_time log_type" command correctly. but the timechart seems to be failing.

Also tried chart but span seems to be not working.

Can any one please help me with ?

source file name -

/var/log/we_accesslog_extsqu_10.10.10.01_20111121_233000_32365
/var/log/mms_export_e_wms_90_10.10.10.02_20111121_232500_09678

| metadata type=sources index="cds_*" NOT index="cds_sysl*" 
| fields source 
| dedup source 
| rex field=source "^(?P<path>[^\_]+)\/(?P<log_type>[^\/1]+)\_(?P<ip>[^\_]+)\_(?P<date>[^\_]+)\_(?P<time>[^\_]+)" 
| eval log_time=round(strptime(time, "%H%M%S"), 0) | convert timeformat="%H:%M:%S" ctime(log_time) 
| eval log_date=round(strptime(date, "%Y%m%e"), 0)  | convert timeformat="%e/%m/%Y" ctime(log_date) 
| fields log_time log_type 

| timechart span=5min count by log_type                ---------  not working

| chart count over log_time by log_type  span=5min    ---------- not working
Tags (2)
1 Solution

Ayn
Legend

What's not working with your second attempt using chart? It looks OK to me, but I wonder whether you checked that your log_time field was actually created correctly?

I simplified your search a bit:

| metadata type=sources index="cds_*" NOT index="cds_sysl*" 
| dedup source 
| rex field=source "^(?<path>[^_]+)/(?<log_type>[^/1]+)_(?<ip>[^_]+)_(?<date>[^_]+)_(?<time>[^_]+)" 
| eval log_time=strptime(time, "%H%M%S")

(note that log_time gets an epoch value. This way you can eval it to _time.)

Then after that you could either use timechart:

| eval _time=log_time | timechart span=5m count by log_type

Or chart, using fieldformat to show a pretty timestamp instead of the actual epoch value.

| bucket log_time span=5m | fieldformat log_time=strftime(log_time, "%H:%M:%S") | chart count over log_time by log_type

View solution in original post

kallu
Communicator

I think the problem is you try to assign "human readable time" into _time while it should be seconds from epoch. Good news is, you already have it from strptime(), ie you don't need those "convert ..." cmds unless you want log_time and log_date for some other purposes.

Maybe this would work better

... | rex field=source "^(?P<path>[^\_]+)\/(?P<log_type>[^\/1]+)\_(?P<ip>[^\_]+)\_(?P<datetime>[^\_]+\_[^\_]+)" | eval _time=strptime(datetime, "%d/%m/%Y_%H:%M:%S") | timechart span=5min count by log_type

If you want use only log_time (ie sum all dates into one) then bucket -cmd could help you. Simply dropping date from above example will make Splunk assume it's current date.

0 Karma

kallu
Communicator

Looks like I waited a bit too long and got second place in this race 🙂

0 Karma

Ayn
Legend

What's not working with your second attempt using chart? It looks OK to me, but I wonder whether you checked that your log_time field was actually created correctly?

I simplified your search a bit:

| metadata type=sources index="cds_*" NOT index="cds_sysl*" 
| dedup source 
| rex field=source "^(?<path>[^_]+)/(?<log_type>[^/1]+)_(?<ip>[^_]+)_(?<date>[^_]+)_(?<time>[^_]+)" 
| eval log_time=strptime(time, "%H%M%S")

(note that log_time gets an epoch value. This way you can eval it to _time.)

Then after that you could either use timechart:

| eval _time=log_time | timechart span=5m count by log_type

Or chart, using fieldformat to show a pretty timestamp instead of the actual epoch value.

| bucket log_time span=5m | fieldformat log_time=strftime(log_time, "%H:%M:%S") | chart count over log_time by log_type
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...