All Apps and Add-ons

eventgen app not generating the data due to the "can't find the time stamp warning".

pavanae
Builder

I was using the sample tutorial 1 as eventgen.conf which is as below :-

[sample_tutorial 1.sample]
mode = replay
sampletype = csv
timeMultiple = 2

backfill = -15m

backfillSearch = index=main sourcetype=splunkd

outputMode = stdout

outputMode = splunkstream

splunkHost = localhost

splunkUser = admin

splunkPass = changeme

outputMode = file
fileName = /tmp/internal.log
token.0.token = \d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2},\d{3,6}
token.0.replacementType = timestamp
token.0.replacement = %Y-%m-%d %H:%M:%S,%f

token.1.token = \d{2}-\d{2}-\d{4} \d{2}:\d{2}:\d{2}.\d{3,6}
token.1.replacementType = timestamp
token.1.replacement = %m-%d-%Y %H:%M:%S.%f

token.2.token = \d{2}/\w{3}/\d{4}:\d{2}:\d{2}:\d{2}.\d{3,6}
token.2.replacementType = timestamp
token.2.replacement = %d/%b/%Y:%H:%M:%S.%f

token.3.token = \d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}
token.3.replacementType = timestamp
token.3.replacement = %Y-%m-%d %H:%M:%S

token.4.token = \d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}
token.4.replacementType = timestamp
token.4.replacement = %Y-%m-%dT%H:%M:%S

Now the below were some of the events in my .csv file which I'd kept in samples directory

Oct 4 08:18:25 xyz.net Oct 4 08:18:06 xyzabc.net 1,2016/10/04 ............
Oct 4 08:19:25 xyz.net Oct 4 08:18:06 xyzabc.net 1,2016/10/04 ............

The below is the error I'm seeing regarding the timestamp

WARNING module='Sample' sample='exported_logs.csv': Can't find a timestamp (using patterns '['\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2},\d{3,6}', '\d{2}-\d{2}-\d{4} \d{2}:\d{2}:\d{2}.\d{3,6}', '\d{2}/\w{3}/\d{4}:\d{2}:\d{2}:\d{2}.\d{3,6}', '\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}', '\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}']') in this event: 'Oct 4 08:18:26 xyz.net Oct 4 08:18:26 xyzabc.net 1,2016/10/04 08:18:26,007801003385,THREAT,url,1,2016/10/04 08:18:26,1.2.3.4.1.111.32,0.0.0.0,0.0.0.0,eserv-unknownURL,x-fwd-for: 11.22.33.44,,web-browsing,vsys1,AM-trust,AM-untrust,ethernet1/2,ethernet1/1,AllThreatsAM,2016/10/04 08:18:26,33919488,1,56014,80,0,0,0x80000,tcp,alert,"vid-io.springserve.com/vd/i?

Based on the above error I understand that the timeformat in the exported_logs in .csv file were different from the eventgen.conf file. So how can i modify the timeformat stanzas in the eventgen.conf file to make it working?

0 Karma
1 Solution

csharp_splunk
Splunk Employee
Splunk Employee

It uses standard strptime format: https://docs.python.org/2/library/datetime.html#strftime-strptime-behavior.

I'd put in a token that looks like:

token.0.token = (\w+\s+\d+\s+\d{2}:\d{2}:\d{2})
token.0.replacementType = timestamp
token.0.replacement = %b %d %H:%M:%S

The number 0 is just an example, pick any free number.

View solution in original post

0 Karma

csharp_splunk
Splunk Employee
Splunk Employee

It uses standard strptime format: https://docs.python.org/2/library/datetime.html#strftime-strptime-behavior.

I'd put in a token that looks like:

token.0.token = (\w+\s+\d+\s+\d{2}:\d{2}:\d{2})
token.0.replacementType = timestamp
token.0.replacement = %b %d %H:%M:%S

The number 0 is just an example, pick any free number.

0 Karma

pavanae
Builder

what does %b %d mean ?

0 Karma
Get Updates on the Splunk Community!

Introducing the Splunk Community Dashboard Challenge!

Welcome to Splunk Community Dashboard Challenge! This is your chance to showcase your skills in creating ...

Built-in Service Level Objectives Management to Bridge the Gap Between Service & ...

Wednesday, May 29, 2024  |  11AM PST / 2PM ESTRegister now and join us to learn more about how you can ...

Get Your Exclusive Splunk Certified Cybersecurity Defense Engineer Certification at ...

We’re excited to announce a new Splunk certification exam being released at .conf24! If you’re headed to Vegas ...