Getting Data In

Why isn't the timestamp being recognized ?

ddrillic
Ultra Champion

We have a case that looks like this -

alt text

So, the events are not broken by the timestamp. Is it because of the underscore between the date and time?

Tags (1)
0 Karma
1 Solution

gjanders
SplunkTrust
SplunkTrust

Splunk has a number of built in time formats, I would recommend you manually parse your timestamps to improve indexing performance and ensure that you have the timestamps consistently recognised.

In this case in your props.conf

[replace_with_your_sourcetype]
TIME_PREFIX = ^
TIME_FORMAT = %Y-%m-%d_%H:%M:%S.%3N

Also refer to configure timestamp recognition. I agree with skoelpin it's probably the _ throwing the default timestamp parsing off.

View solution in original post

0 Karma

gjanders
SplunkTrust
SplunkTrust

Splunk has a number of built in time formats, I would recommend you manually parse your timestamps to improve indexing performance and ensure that you have the timestamps consistently recognised.

In this case in your props.conf

[replace_with_your_sourcetype]
TIME_PREFIX = ^
TIME_FORMAT = %Y-%m-%d_%H:%M:%S.%3N

Also refer to configure timestamp recognition. I agree with skoelpin it's probably the _ throwing the default timestamp parsing off.

0 Karma

ddrillic
Ultra Champion

Btw, this by itself did it ; -) with no line breaking

0 Karma

ddrillic
Ultra Champion

Gorgeous - thank you @garethatiagg

skoelpin
SplunkTrust
SplunkTrust

Remember that this is a start and you should apply line breaking to your props.conf aswell.. This will take some load off your indexers

gjanders
SplunkTrust
SplunkTrust

The documentation also covers line breaking

This is a slightly more advanced topic but potentially worth discussing here, the default for the MAX_EVENTS in props.conf is 256 lines at the time of writing, if you always have single line events you could switch SHOULD_LINEMERGE=false on and never line merge (this entry would go in the props.conf)

Alternatively if you have very large events (>1000 lines) you could make a more complicated LINE_BREAKER setting and change SHOULD_LINEMERGE to false such as:

[replace_with_your_sourcetype]
TIME_PREFIX = ^
TIME_FORMAT = %Y-%m-%d_%H:%M:%S.%3N
SHOULD_LINEMERGE = false
LINE_BREAKER = ([\r\n]+)\d{4}-\d{2}-\d{2}_\d

Of course the above would require testing, and it would only make sense if you have larger multi-line events (I've used examples like the above for events >1000 lines), doing this can remove pressure from the aggregation queue as the merging will move to the parsing queue...

skoelpin
SplunkTrust
SplunkTrust

Most likely the underscores are throwing it off. What's your props.conf look like?

0 Karma
Get Updates on the Splunk Community!

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...