Getting Data In

How to extract the date from a filename at index-time to use as _time?

WXY
Path Finder

I want to extract the year, month and day from the file name. The file name format is: aa_1_20180701.csv OR aa_2_20180702.csv
I configured it in /etc/apps/myapp/local/datetime.xml as follows:(I use a Splunkforwarder to forward data,and I also configue datetime.xml,and put the datetime.xml in $SplunkForwarder/etc/apps/myapp/local/datetime.xml)

<datetime>
<define name="_dateconfim" extract="year, month, day">
 <text>
 <![CDATA[source::.*?((\d{4})(\d{2})(\d{2}))]]></text>
 </define>

<timePatterns>
      <use name="_dateconfim"/>
</timePatterns>
<datePatterns>
<use name="_dateconfim"/>
</datePatterns>

</datetime>

My sourcetype configue is:

[mysource]
DATETIME_CONFIG = /etc/apps/myapp/local/datetime.xml
INDEXED_EXTRACTIONS = csv
NO_BINARY_CHECK = true
SHOULD_LINEMERGE = false
category = Structured
disabled = false
pulldown_type = true

But the time I got was always modification time. alt textalt text
what should I do ?
I wang to get time which in the filename

0 Karma

yannK
Splunk Employee
Splunk Employee

It looks like you are on the good track.
- maybe a typo somewhere
- make sure that you put the props.conf and custom_datatime.xml in the indexers (or heavy forwarders if any)

there is a bit of documentation on the formats here : https://docs.splunk.com/Documentation/Splunk/7.0.4/Data/Configuredatetimexml

0 Karma

yannK
Splunk Employee
Splunk Employee

A small remark on the datetime .

Usually, I avoid reusing the name datetime.xml for my file, because of the merging process between apps.
I would recommend to create your own file mydatetime.xml to make sure that the config is not picking another default one.

0 Karma

niketn
Legend

@WXY I am not sure if you have tried defining Timestamp extraction (only Time part) from your events and Splunk should be able to pull the date from Filename. Refer to an older answer: https://answers.splunk.com/answers/557841/how-to-extract-date-from-filename-and-add-it-with.html

If you can provide some sample events with Time/Time format we would be able to assist you better.

____________________________________________
| makeresults | eval message= "Happy Splunking!!!"
0 Karma

AKG1_old1
Builder

Hi,

did you managed to get it working ? I am having same issue.

Thanks
Ankit

0 Karma

DalJeanis
SplunkTrust
SplunkTrust

@agoyal - please do not revive dead posts - it is more effective to ask your new question and put a link to the old post in your own, new question.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...