How to configure Splunk to recognize the time format and extract the _time field from the source file name?

[vasanthmss]

Hi Experts,

I don't have a time stamp field in any of my events. As of now, the default system time is added as _time.

File comes by hourly with the file name (filename_201503121000.csv). [filename_YYYYMMDDHHSS.csv ]. Now I am trying to extract the _time from the source file name. I created a transformation and then calculated _time based on source.

It seems like _time has changed,

Earlier: search duration is <2015-Mar-12 00:00 to 2015-Mar-12 12:00>

Filename_201503121000.csv [ 12-Mar-2015 10:00] comes at 11.10 so all the events are looking like this,

03/12/2015 11.10 <event 1>
03/12/2015 11.10 <event 2>
03/12/2015 11.10 <event 3>
...
03/12/2015 11.10 <event n>

After the change, the _time : search duration is

<2015-Mar-12 00:00 to 2015-Mar-12 12:00>

03/12/2015 10.00 <event 1>
03/12/2015 10.00 <event 2>
03/12/2015 10.00 <event 3>
...
03/12/2015 10.00 <event n>

When I select the duration <2015-Mar-12 10:00 to 2015-Mar-12 10:59> the records are not available. Seems like _time has been changed and it's not possible to search using that updated _time.

Some times there would be some delay in third party system which posts the file so it's required to change the time. Help me to change _time based on the file name(source).

Thanks in advance.

Cheers!

[woodcock · Accepted solution]

This can be done with datetime.xml. This should work for you:

<datetime>
<define name="datefromfile" extract="year, month, day">
   <text><![CDATA[source::.*?_(\d{4})(\d{2})(\d{2}]]></text>
</define>
<define name="timefromfile" extract="hour, second">
   <text><![CDATA[source::.*?_\d{8}(\d{2})(\d{2}]]></text>
</define>
<timePatterns>
   <use name="timefromfile"/>
</timePatterns>
<datePatterns>
   <use name="datefromfile"/>
</datePatterns>
</datetime>