Time not parsed correct suddenly

[davidjehoul]

I have lines of data that looks like this (1 line) in the file source="C:\Temp\testResultLog.csv":

RT0963-01,7/02/2013 13:33:22,19/04/2013 11:13:03,0,R_1812,0,Netscape3.0,0,0,0,172.21.0.132,172.21.0.132,ohm-web-7.9.5 (d921a - 2013-05-02 13:30:00),20130502_133229,2/05/2013 20:00:02,2/05/2013 20:00:08,6,True,DAVIDJ-3500,x86,4

For the file containing these lines I have the following in the props.config file:

[testResultLog]  
CHECK_FOR_HEADER = true  
KV_MODE = none  
MAX_TIMESTAMP_LOOKAHEAD = 20  
NO_BINARY_CHECK = 1  
SHOULD_LINEMERGE = False  
TIME_FORMAT = %d/%m/%y %H:%M:%S  
TIME_PREFIX = \d{8}_\d{6},  
pulldown_type = 1

This should extract the time as indicated in bold in the data. For times before may, Splunk extracted this correct; however from may, it parses the date in the above line as the 5th of february, whereas it should parse it as the second of may.

An example of a line that was parsed correct:

BT01-02,18/03/2008 9:26:09,19/04/2013 11:11:16,0,R_1812,0,Netscape3.0,0,0,3,172.21.0.120,172.21.0.120,ohm-web-8.0.0-SNAPSHOT (c2601 - 2013-04-28 21:01:16),20130429_110040,29/04/2013 11:00:43,29/04/2013 11:02:34,111,True,DAVIDJ-3500,x86,4

Thanks for the help!

David

[kristian_kolb · Accepted solution]

Hi there,

It seems like you have marked (in bold) the wrong part of the message, given the TIME_PREFIX declaration. Splunk will start looking for a timestamp immediately after the TIME_PREFIX, which in this case is dddddddd_dddddd,.

Also, your TIME_FORMAT is in part wrong. You could try to change your it to;

TIME_FORMAT = %e/%m/%Y %H:%M:%S

where %e is days 1-31, instead of %d, which is 01-31
and %Y which is YYYY, instead of %y, which is YY

See http://www.strftime.net for more info.

Hope this helps,

Kristian

[davidjehoul]

Indeed, I marked the wrong part. Sorry for that! Thanks for the answer!

[Ayn]

This is probably due to that you specify that Splunk should look for the 2-digit day of the month (%d) instead of the 1-digit day (%e), which is what you'll find in your logs at the beginning of the month. In other words, Splunk would fully understand the string "30/04" when you use "%d/%m", but not "1/05" (it would have to be "01/05" to be matched by the format string). Also, you have "%y" in your format string - %y is the year without century so the format string would match "13" but not "2013".

Due to this I'm pretty sure that your own format string has never worked, so Splunk has relied on its own default ones for finding valid timestamps, and these by default look for timestamps in US format where month comes before date.

A format string that would probably work better would look like

TIME_FORMAT = %e/%m/%Y %H:%M:%S

[kristian_kolb]

NOT AGAIN!!! I'll still leave my posting because of the TIME_PREFIX issue.