Getting Data In

Timestamp Extraction

royimad
Builder

I have a log with multiple lines that contains several timestamps. When monitoring the logs splunk is split them into several events , I just need to split on the first timestamp of the begining of the lines with a specific format without considered my log as several events:

I have used TIME_FORMAT and LINE_BREAKER but this isn't help- maybe TIME_PREFIX will help but don't know how to figure it out, MY TIME is only the begin of a newline.

props.conf
----------

[log4j]
NO_BINARY_CHECK = 1
TIME_FORMAT = %a %b %e %Y %k:%M:%S,%3 %Z
LINE_BREAKER = \w{3}\s\w{3}\s\d{2}\s\d{4}\s\d{2}:\d{2}:\d{2}

MYLOG:
-------

<EVENT1>
Thu Feb 28 2013 08:58:39,024 EST ERROR .persistence.exceptions.DatabaseException
Internal Exception: java.sql.SQL
bind => [SH90054, 2013-02-27 10:20:16.425, 2013-02-27 10:20:16.425, null, null, F, MD80828, null]
Query: InsertObjectQuery()
at org...
<EVENT2>

Thu Feb 28 2013 08:60:39,024 EST WARN XYZ

0 Karma

rsennett_splunk
Splunk Employee
Splunk Employee

To grab the first time stamp:
MAX_TIMESTAMP_LOOKAHEAD will tell Splunk how far to look before identifying the timestamp.
In other situations, you might use this to take the 2nd timestamp by setting MAX_TIMESTAMP_LOOKAHEAD = 40.

If you would prefer to use LINE_BREAKER, you'll want to review this:
http://docs.splunk.com/Documentation/Splunk/5.0.2/Data/Indexmulti-lineevents

LINE_BREAKER requires a capturing group that defines the newline and carriage returns. It's a little difficult to work with because you need to be sure that you account for the accurate pattern.

You might want to try something like this:

[log4j]
BREAK_ONLY_BEFORE=\w{3}\s\w{3}\s\d{2}\s\d{4}\s\d{2}:\d{2}:\d{2}
MAX_TIMESTAMP_LOOKAHEAD=10
NO_BINARY_CHECK=1
SHOULD_LINEMERGE=true

With Splunk... the answer is always "YES!". It just might require more regex than you're prepared for!
Get Updates on the Splunk Community!

Adoption of RUM and APM at Splunk

    Unleash the power of Splunk Observability   Watch Now In this can't miss Tech Talk! The Splunk Growth ...

Routing logs with Splunk OTel Collector for Kubernetes

The Splunk Distribution of the OpenTelemetry (OTel) Collector is a product that provides a way to ingest ...

Welcome to the Splunk Community!

(view in My Videos) We're so glad you're here! The Splunk Community is place to connect, learn, give back, and ...