Looking for some guidance on non-standard date/time parsing…
We have a customer that has logs without years
We're going round and round trying to get them to put the year into their logs, but are still getting some pressure to make it work as is
I've looked at using strptime via TIME_FORMAT in props.conf, but this raises some questions
(1) How to account for the comma then millisecs? - I stripped it in the outside tests (subseconds, possibly %3N?)
(2) Strptime testing outside of splunk sets year to 1900 if not specified
(3) Can the year be set to a default or current value in props or transforms?
A sample line is > 1800 characters
when tested manually against a pattern, splunk seems to parse the date OK and substitute the current year for the missing year
when tested against a source file, with lines beginning with the same values, splunk fails
[splunk@box1 samples]$ splunk test dates applog.log
Using logging configuration at /opt/instance/splunk/etc/log-cmdline.cfg.
Unable to parse 'applog.log'
[splunk@box1 samples]$ splunk test dates "23:59:48,243 01/11 INFO aaaStateLogger"
Using logging configuration at /opt/instance/splunk/etc/log-cmdline.cfg.
From: 23:59:48,243 01/11 INFO aaaStateLogger
Parsed: Tue Jan 11 23:59:48 2011
UTC Time: 1294808388
Time Region: 0-13
Date Region: 13-18
Subseconds: 0.243
[splunk@box1 samples]$
I have posted a partial extract of a few lines below
Sample trimmed data:
23:57:44,491 01/11 INFO aaaStateLogger - aaaTerminationRequest[aaaState=,menuCode...
23:57:55,459 01/11 INFO aaaStateLogger - aaaTerminationRequest[aaaState=,menuCode...
23:57:57,233 01/11 INFO aaaStateLogger - aaaTerminationRequest[aaaState=,menuCode...
23:57:57,730 01/11 INFO aaaStateLogger - aaaTerminationRequest[aaaState=,menuCode...
23:58:07,700 01/11 INFO aaaStateLogger - aaaTerminationRequest[aaaState=,menuCode...
23:58:10,515 01/11 INFO aaaStateLogger - aaaTerminationRequest[aaaState=,menuCode...
23:58:50,544 01/11 INFO aaaStateLogger - aaaTerminationRequest[aaaState=,menuCode...
23:59:07,033 01/11 INFO aaaStateLogger - aaaTerminationRequest[aaaState=,menuCode...
23:59:16,737 01/11 INFO aaaStateLogger - aaaTerminationRequest[aaaState=,menuCode...
23:59:48,243 01/11 INFO aaaStateLogger - aaaTerminationRequest[aaaState=,menuCode...
... View more