Since March 13th 2011 GMT, Splunk no longer properly parses my epoch timestamps (seconds since 1970)

[carasso]

Since March 13th 2011 GMT, Splunk no longer properly parses my epoch timestamps (seconds since 1970).

For example, I am indexing the following log file :

[root@beefysup01 splunk]# ls -l /test/epoch/asterisk.log 
-rw-r--r-- 1 root root 236 Mar 15 13:24 /test/epoch/asterisk.log

Contents of the file :

1300218018|NONE|NONE|NONE|CONFIGRELOAD|
1300218018|NONE|NONE|NONE|CONFIGRELOAD|
1300218957|NONE|NONE|NONE|QUEUESTART|
1300218958|NONE|NONE|NONE|CONFIGRELOAD|
1300219007|NONE|NONE|NONE|QUEUESTART|
1300219007|NONE|NONE|NONE|CONFIGRELOAD|

We expect Splunk to assign the following time stamps to these events :

Tue Mar 15 12:40:18 PDT 2011 : 1300218018|NONE|NONE|NONE|CONFIGRELOAD|
Tue Mar 15 12:40:18 PDT 2011 : 1300218018|NONE|NONE|NONE|CONFIGRELOAD|
Tue Mar 15 12:55:57 PDT 2011 : 1300218957|NONE|NONE|NONE|QUEUESTART|
Tue Mar 15 12:55:58 PDT 2011 : 1300218958|NONE|NONE|NONE|CONFIGRELOAD|
Tue Mar 15 12:56:47 PDT 2011 : 1300219007|NONE|NONE|NONE|QUEUESTART|
Tue Mar 15 12:56:47 PDT 2011 : 1300219007|NONE|NONE|NONE|CONFIGRELOAD|

Instead, Splunk is assigning the same time stamp to all events : Tue Mar 15 13:24:00 PDT 2011 - This is the modtime of the file and not accurate in the context of each event.

Strangely, there are no DateParserVerbose errors recorded in splunkd.log.

[carasso · Accepted solution]

In Splunk's datetime.xml, the regular expression for parsing epoch times, assumes values from 2002 through to March 13th 2011. Those values started with 10,11,12. On Sunday March 13th 2011 07:06:40 GMT, the seconds since 1970 became 1300000000, starting with 13.

This will be fixed in 4.2.1.

There are two possible fixes. The first involves editing an xml file and will resolve the problem for all data sources. The second doesn involves editing props.conf and will fix things for whatever source/sourcetype/host you change.

---

1) The fix is to modify $SPLUNK_HOME/etc/datetime.xml, around line 200. Change the _utcepoch regex to the following:

<define name="_utcepoch" extract="utcepoch, subsecond">
    <!-- update regex before 2017! 🙂 -->
    <text><![CDATA[((?<=^|[\s#,"=([\|{])(?:1[012345]|9)\d{8}|^@[\da-fA-F]{16,24})(?:.?(\d{1,6}))?(?![\d(])]]></text>
</define>

Make sure to make a backup of your file first.

---

2) Alternatively, for your sources that use epoch time, explicitly specify a strptime format in props.conf, by using TIME_FORMAT and TIME_PREFIX fields. For example, see http://answers.splunk.com/questions/8428/how-do-i-recognize-a-time-in-epoch-seconds

[jplumsdaine22]

In the datetime.xml for 6.3.2 I can see the following :

<define name="_utcepoch" extract="utcepoch, subsecond">
      <!-- update regex before '2017' -->

But the regex (it is: (?:1[012345]|9)) looks good to 1599999999, or the 13th of September 2020

So hopefully we'll see an update in the next couple of years

[acharlieh]

Not sure how you're doing epoch time conversion @jplumsdaine22 but 1599999999 is Sun, 13 Sep 2020 12:26:39 GMT.

[jplumsdaine22]

Derp I copied the wrong line from my converter. Updated my original comment