Hi everybody, is it possible to teach a custom datetime.xml that my subsecond field is only two digit long?
I have a log file with two different timestamp formats, one of which is not automatically recognized. Here's an example:
00000000000000000001289525909952;2010-11-12 02:38:32:47;P2LIV ....
00000000000000000001289525916501;2010111202383591;YT00B ....
so I had to configure a custom datetime.xml which is working fine, except I am not able to let it extract the subseconds: Splunk always interprets them as zero.
Here's my custom datetime.xml:
<datetime>
<define name="_combdatetime_consMSG1" extract="year, month, day, hour, minute, second, subsecond">
<!-- ... 2010-11-12 02:38:36:90 ...' -->
<text><![CDATA[(\d+)-(\d+)-(\d+)\s(\d+):(\d+):(\d+):(\d+)]]></text>
</define>
<define name="_combdatetime_consMSG2" extract="year, month, day, hour, minute, second, subsecond">
<!-- ... 2010111202383691 ...' -->
<text><![CDATA[(\d{4})(\d{2})(\d{2})(\d{2})(\d{2})(\d{2})(\d{2})]]></text>
</define>
<timePatterns>
<use name="_combdatetime_consMSG1"/>
<use name="_combdatetime_consMSG2"/>
</timePatterns>
<datePatterns>
<use name="_combdatetime_consMSG1"/>
<use name="_combdatetime_consMSG2"/>
</datePatterns>
</datetime>
And this is how Splunk web displays those two timestamps
11/12/10 2:38:32.000 AM
11/12/10 2:38:35.000 AM
I've tried to manually modify the source file to put a trailing 0 after the timestamp (to bring it to a millis format) and changed the regexes accordingly, but had no luck (after a full restart, too) Anybody has an idea?
Thanks, Paolo
Hi Paolo,
You may be better off using TIME_FORMAT in props.conf rather than changing datetime.xml. Its handling of subseconds and subsecond precision is covered on another answers post:
http://answers.splunk.com/questions/1946/time-format-and-subseconds
D'oh! I completely missed that aspect of what you were explaining.
Thanks dwaddle, I've tried the %2N to specify only two digits (or 9, on other CICS logs), but then I would have to stick with one format only and "lose" the second.
As a sidenote: MAX_TIMESTAMP_LOOKAHEAD is set high enough for Splunk to read the whole timestamp.