Getting Data In

Pull Timestamp from Field in Raw Data

rdschmidt
Explorer

Can anyone tell me how to configure my Props.conf to use a defined field "Event_Time" (Which is in Epoch Time) for the timestamp (_time) instead of pulling the time when the file was saved?

Currently i have this configured:

TIME_FORMAT = %b %d %H:%M:%S ctime(Event_Time)
MAX_TIMESTAMP_LOOKAHEAD = 32
TZ = US/Central

Thanks,

Riley

Tags (3)
0 Karma

ryainad
Explorer

Hi, Riley.

I have a question regarding Huawei CSOFTX3000 CDRs. What do you use to decode CDR file? I search for solution, and I found only this splunk application from dmillis. But still I dont understand how to use it to decode CDR files.

Thank you in advance.

0 Karma

ryainad
Explorer

I got this Huawei CDR sample file. But I don't know know what type of the file is it. (250 byte or 350 byte or 450 byte) Could you please tell me how to know it?

0 Karma

rdschmidt
Explorer

The above time_prefix did fix our TimeStamp issues on our AAA records. We are still working on the CSOFTX3000. The issue is we are using 450 byte CDRs and the app is built for 350 byte.

linu1988
Champion

TIME_FORMAT=%s
TIME_PREFIX= \d{7}\|\d{6}\|\d\|
MAX_TIMESTAMP_LOOKAHEAD = 10

could you try setting props.conf and index new data?

0 Karma

Ayn
Legend

http://docs.splunk.com/Documentation/Splunk/5.0.2/Data/Configuretimestamprecognition
http://docs.splunk.com/Documentation/Splunk/latest/Data/HowSplunkextractstimestamps

You don't extract timestamps from fields, because field extractions happen at a much later stage (and for most fields doesn't happen at index-time at all).

I took your sample event and threw it into RegExr (http://gskinner.com/RegExr/ ) and came up with a TIME_PREFIX regex that should work for you:

TIME_PREFIX = ^(?:[^|]*\|){34}

After that you can just use "TIME_FORMAT = %s" because it's an ordinary epoch timestamp.

rdschmidt
Explorer

Here is my search : x@wireless.com | convert ctime(Event_Time) as TIME

I just want to make the TIME field automatically show up as the timestamp.

Thanks,

Riley

0 Karma

rdschmidt
Explorer

Raw Data:
213|2|0|1|0|x||x.x.x.x|x@wireless.com|0782A8FC|07784722|0|0|x.x.x.x|x.x.x.x|x.x.x.x|0083|0|0|0|59|0|0|0|0|0|2|1|3|0|0|9747835|197309|0|1373498910|2019|0|0|0|0|0|211465|0|0|10|0|0|0|0|0||10026|2|0|1|541|3539|6668|1|07784722|0|0|0|24|x|0104000102040001|875560960||0|0|0|0||1373498539||0|311650|0|0|0|0|0|0|0|0|0||0|0|0|-1||||

timestamp: 7/10/13 6:54:51.000 PM

Derived Fields:
| Event_Time=1373498910 | TIME=07/10/2013 18:28:30

0 Karma

linu1988
Champion

Please use

TIME_FORMAT=%s
TIME_PREFIX= (regex)
MAX_TIMESTAMP_LOOKAHEAD = 32

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...