Splunk Search

How to configure Splunk to extract the correct year from the date stamp in my sample log?

daniel_augustyn
Contributor

I am not sure how to fix the date extraction from a raw log which is done by default by Splunk. Splunk extracts date by default and it's not doing the year correctly.

This is the raw log:

Jun 21 00:00:32 10.20.14.12 Jun 20 17:00:32 : 2016/06/20 17:00:32 PDT,1,7016505,L2 Poll Failed,0,10596,,LAB,10.18.8.1,,L2 Poll failed to read hosts from LAB.

And this is date that is getting extracted:

6/20/12 5:00:32.000 PM

Anyone knows how to fix it?

0 Karma
1 Solution

ryanoconnor
Builder

Do you have any way of modifying the format of these logs? Ideally the first portion of your log would be a valid timestamp. In this case Splunk is getting confused because it sees a portion of a valid date at the beginning of the log.

You may be able to work around it using the following, assuming this is your timestamp:

2016/06/20 17:00:32 PDT

You'll need to configure a props.conf file to recognize this.

[your_sourcetype] 
TIME_PREFIX = ^.*\s:\s
MAX_TIMESTAMP_LOOKAHEAD = 24
TIME_FORMAT = %Y/%m/%d %T %Z

http://docs.splunk.com/Documentation/Splunk/6.4.1/Data/Configuretimestamprecognition

View solution in original post

ryanoconnor
Builder

Do you have any way of modifying the format of these logs? Ideally the first portion of your log would be a valid timestamp. In this case Splunk is getting confused because it sees a portion of a valid date at the beginning of the log.

You may be able to work around it using the following, assuming this is your timestamp:

2016/06/20 17:00:32 PDT

You'll need to configure a props.conf file to recognize this.

[your_sourcetype] 
TIME_PREFIX = ^.*\s:\s
MAX_TIMESTAMP_LOOKAHEAD = 24
TIME_FORMAT = %Y/%m/%d %T %Z

http://docs.splunk.com/Documentation/Splunk/6.4.1/Data/Configuretimestamprecognition

daniel_augustyn
Contributor

I added this to the props.conf stanza on the search head under system/local/ and it didn't help. I am still getting logs with wrong year in them.

0 Karma

MuS
Legend

@daniel_augustyn , theses setting need to be done where the parsing is happening, usually an indexer or a heavyweight forwarder. See this http://wiki.splunk.com/Where_do_I_configure_my_Splunk_settings%3F to learn more about this topic.

daniel_augustyn
Contributor

Awesome, it totally fixed it.

0 Karma
Get Updates on the Splunk Community!

Stay Connected: Your Guide to May Tech Talks, Office Hours, and Webinars!

Take a look below to explore our upcoming Community Office Hours, Tech Talks, and Webinars this month. This ...

They're back! Join the SplunkTrust and MVP at .conf24

With our highly anticipated annual conference, .conf, comes the fez-wearers you can trust! The SplunkTrust, as ...

Enterprise Security Content Update (ESCU) | New Releases

Last month, the Splunk Threat Research Team had two releases of new security content via the Enterprise ...