Getting Data In

Why are some events being indexed with the wrong timestamp?

alexanderadler
New Member

Hi! I have a big Splunk enterprise environment, but I'm experiencing a strange issue where some events are losing part of their timestamp. These are the timestamps for the events

2019-01-08 07:05:32,776 StatisticMessage ,
2019-01-08 07:05:33,166 StatisticMessage , 
2019-01-08 07:05:33,401 StatisticMessage , 

alt text
and this is the props.conf file that is deployed.

[NameOfSourceType]
MAX_TIMESTAMP_LOOKAHEAD = 60
TIME_FORMAT = %Y-%m-%d %H:%M:%S,%3N
SHOULD_LINEMERGE = false
LINE_BREAKER = ([\n\r]+)\d{4}\-\d{2}\-\d{2}\s\d{2}\:\d{2}\:\d{2}\,\d{3}
TRUNCATE = 1999999

But some events are getting part of their time scrambled as shown below.

alt text

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Have you seen any DateParserVerbose warnings in splunkd.log about this source/sourcetype?

---
If this reply helps you, Karma would be appreciated.
0 Karma

lakshman239
SplunkTrust
SplunkTrust

I assume you want to extract the date and time from the first 24 chars, so try the below:
//limiting the lookahead, adding prefix and changing line breaker to be positive lookahead

[NameOfSourceType]
MAX_TIMESTAMP_LOOKAHEAD = 24
TIME_PREFIX = ^
TIME_FORMAT = %Y-%m-%d %H:%M:%S,%3N
SHOULD_LINEMERGE = false
LINE_BREAKER = ([\n\r]+)(?=\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2}\,\d{3}\s)
TRUNCATE = 999999

0 Karma

alexanderadler
New Member

Hi! Tried this but still the same outcome.

0 Karma

lakshman239
SplunkTrust
SplunkTrust

send sample file with a few events [ pls remove/mask sensitive data]

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...