Getting Data In

Issue with timestamp extraction

vishaltaneja070
Motivator

Hello

I have used the below setting in props, but the first event is not able to extract the timestamp:

[sourcetype]
SHOULD_LINEMERGE=false
NO_BINARY_CHECK=true
TIME_FORMAT=%m/%e/%Y %H:%M:%S %p
TIME_PREFIX=[\r\n]+
MAX_TIMESTAMP_LOOKAHEAD=50
LINE_BREAKER=([\r\n]+)(\/\/\s[\-]+\s\/\/)
BREAK_ONLY_BEFORE=([\r\n]+)(\/\/\s[\-]+\s\/\/)
TRUNCATE=0
SEDCMD-EXTRALINES=s/\/\/\s[\-]+\s\/\///g

Event Sample

6/12/2019 4:12:40 AM (6/12/2019 8:12:40 AM) LogLevel=Information
Process_ID=6175, Thread.Thread_ID=8588 Thread.Thread_Name=()
Message=c
Extended Properties
Origin=hxhdgjshjs
TrackingToken=1c1fb75e-jdhhdd-jdhjdhd

// ----------- //
6/12/2019 4:13:40 AM (6/12/2019 8:13:40 AM) LogLevel=Information
Process_ID=6175, Thread.Thread_ID=85883 Thread.Thread_Name=()
Message=c
Extended Properties
Origin=hxhdgjshjs
TrackingToken=1c1fb75e-jdhhdd-jdhjdhd

// ----------- //
6/12/2019 4:14:40 AM (6/12/2019 8:14:40 AM) LogLevel=Information
Process_ID=6175, Thread.Thread_ID=85488 Thread.Thread_Name=()
Message=c
Extended Properties
Origin=hxhdgjshjs
TrackingToken=1c1fb75e-jdhhdd-jdhjdhd

// ----------- //

Also I don't want to use SHOULD_LINEMERGE=true. Thanks!

0 Karma

woodcock
Esteemed Legend

The problem is that the SEDCMD is removing the lines that you are depending on and need for LINE_BREAKER. You do not need the SEDCMD because the LINE_BREAKER will remove them. Also, I am not sure what will happen when your LINE_BREAKER has multiple capture groups so I would use this:

[<Your Sourcetype Here - YOU MUST SET THIS>]
SHOULD_LINEMERGE=false
LINE_BREAKER=([\r\n\s]+\/\/\s+[\-]+\s+\/\/[\r\n\s]*)
NO_BINARY_CHECK=true
TIME_PREFIX=^
TIME_FORMAT=%m/%d/%Y %H:%M:%S %p
MAX_TIMESTAMP_LOOKAHEAD=22
TRUNCATE=999999

If you are doing a sourcetype override/overwrite, you must use the ORIGINAL value, NOT the new value, then you must deploy this to the first full instance(s) of Splunk that handles the events (usually either the HF tier, if you use one, or else your Indexer tier), restart all Splunk instances there, send in new events (old events will stay broken), then test using _index_earliest=-5m to be absolutely certain that you are only examining the newly indexed events.

0 Karma

maciep
Champion

I would probably do something like this, which should start each event at the timestamp. of course, you can use sed to get rid of the line separators if you don't want them

[your_sourcetype]
LINE_BREAKER = ([\r\n]+)(?=\d+\/\d+\/\d+\s+\d+:\d+:\d+)
TIME_PREFIX = ^
TIME_FORMAT = %m/%d/%Y %H:%M:%S %p
MAX_TIMESTAMP_LOOKAHEAD = 24
SHOULD_LINEMERGE = false
0 Karma

493669
Super Champion

Try

TIME_PREFIX=^

since timestamp is at the start of line.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...