Getting Data In

How to extract date field from the filename in Splunk and assign _time value to that in indexing phase

snehalk
Communicator

Hello Everyone,

I have text file 20170701.txt where 2017-year, 07-month and 01-date.

This file is coming from the universal forwarder, below is my inputs.conf (C:\Program Files\SplunkUniversalForwarder\etc\system\local)

[monitor://C:\sampletestfile\*]
index=test
sourcetype=largefile
crcSalt = <SOURCE>

In the heavy forwarder side i wrote the datetime.xml and props.conf (C:\Program Files\Splunk\etc\apps\myapp\local)

  <define name="_mydatetime" extract="year, month, day">
    <text><![CDATA[source::.*?_(\d{4})(\d{2})(\d{2}).txt]]></text>
 </define>
 <timePatterns>
    <use name="_mydatetime"/>
 </timePatterns>
 <datePatterns>
    <use name="_mydatetime"/>
 </datePatterns>
 </datetime> 

props.conf

[largefile] 
DATETIME_CONFIG = /etc/apps/myapp/local/datetime.xml

After the doing changes i have restarted both UF and HF.

The problem is am not getting "20170701" date as indexing time in Splunk.

And in Splunkd.log am getting below error

07-05-2017 16:13:54.893 +0530 ERROR AggregatorMiningProcessor - Uncaught exception in Aggregator, skipping an event: Error parsing regex XML file: C:\Program Files\Splunk\etc\apps\myapp\local\datetime.xml - Couldn't find 'timePatterns' in config data for AggregatorProcessor. - data_source="C:\sampletestfile\20170701.txt"

Can any one please guide me where am wrong.

Thanks in Advance

jplumsdaine22
Influencer

There's no tag at the start of your xml file, which is probably causing the whole thing to be skipped.

0 Karma

snehalk
Communicator

Hello jplumsdaine22,
Thanks for reply, after adding tag in code, still am getting same error. could you please help me here?

0 Karma

gregbo
Communicator

Did you ever get help on this?

0 Karma

woodcock
Esteemed Legend

Why are you using Heavy Forwarder here? It seems to add no benefit and therefore makes no sense.

0 Karma

snehalk
Communicator

Hello woodcock,

Yes,in current scenario there is no need, but we have this as current environment, and also we have some files where we are using parsing. but not for this source.

0 Karma
Get Updates on the Splunk Community!

Introducing the 2024 SplunkTrust!

Hello, Splunk Community! We are beyond thrilled to announce our newest group of SplunkTrust members!  The ...

Introducing the 2024 Splunk MVPs!

We are excited to announce the 2024 cohort of the Splunk MVP program. Splunk MVPs are passionate members of ...

Splunk Custom Visualizations App End of Life

The Splunk Custom Visualizations apps End of Life for SimpleXML will reach end of support on Dec 21, 2024, ...