Getting Data In

Set timestamp based on file source path

gelica
Communicator

Hi all,

I'm trying to set the timestamp for events from my source. My paths look like this:

C:\Users\angeliga\Filer\336033\gelica_2013-03-06_13-48-45\Server\file_to_index.txt

I have read some answers on this subject here at splunk-base and on some other places.
The suggestions that I've come across are to copy datetime.xml and modify it (from this splunk-base answer), or to do it in transforms.conf (from this splunk-base answer)

But I can't get it to work!

It seems to me that the easiest way would be to use transforms.conf, but I can't figure out how to set the field correctly..

I've also followed the exmples on how to modify datetime.xml, but when it looks like below, I get no events of that my_src_type! To figure out if I did something wrong when editing datetime.xml, I tried to just copy (no editing) it into my local folder and then set DATETIME_CONFIG = /etc/system/local/datetime.xml but it doesn't matter, I still get no events of my_src_type...

[my_src_type]
DATETIME_CONFIG = /etc/system/local/datetime.xml
other sourcetype stuff...

I would also be able to extract the date, but I'm thinking that it would be the same approach?

I hope someone can help me with this, it is very frustrating that I'm not able to make it work.

0 Karma
1 Solution

gelica
Communicator

In case someone else have this problem, I didn't manage to get it working by using datetime.xml..
Instead I used EVAL in props.conf:

EVAL-_time=strptime(file_name, "%Y-%m-%d_%H-%M-%S")

Probably not the most efficient way to do this, but it works for me for now.

I'm still open to try another way if anyone has any solution.

View solution in original post

gelica
Communicator

In case someone else have this problem, I didn't manage to get it working by using datetime.xml..
Instead I used EVAL in props.conf:

EVAL-_time=strptime(file_name, "%Y-%m-%d_%H-%M-%S")

Probably not the most efficient way to do this, but it works for me for now.

I'm still open to try another way if anyone has any solution.

gelica
Communicator

@crt89 I'm not sure, and I'm not able to test since I'm not in that project anymore.
The only thing that comes to my mind is that maybe file_name isn't what you think it is, have you double checked that?
Good luck

0 Karma

crt89
Communicator

Hi @gelica. I am currently having this same problem. I want the timestamp of the events of my log to be the timestamp on its filename. I see you have managed to do this and I have a question in your config. I tried your config here's mine: EVAL-_time=strptime(file_name, "%m-%d-%Y") and my filename is this: MTYP0-09-26-2013.log. I can't get the timestamp of the file. Hope you can help me on this

0 Karma

Ayn
Legend

Did you check splunkd.log for any errors related to this time extraction? The timestamp processor is usually pretty good at telling why it's failing for one reason or another. Also I'm assuming you've read this docs page: http://docs.splunk.com/Documentation/Splunk/latest/Data/HowSplunkextractstimestamps

royimad
Builder

Add this line to props.conf file and extract the date from the directory name

EXTRACT-sourcefields = \Users\angeliga\Filer\336033\gelica_(?<the_date>.*)\Server\file_to_index.txt in source
0 Karma

gelica
Communicator

Thanks for your answer, but I'm looking for a way to do this at index time, and make it the timestamp of the events in order to be able to use timechart and stuff easily.

0 Karma
Get Updates on the Splunk Community!

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...