Getting Data In

Event Time Stamp: Day and Month Switched by Splunk

intelli2019
New Member

Hi,
I have 1 months worth of logs I am uploading to Splunk cloud manually as a trial for when our Enterprise license comes in.
Splunk recognises most of the time stamps correctly and assigns the correct _time to them however some have the DD and MM switched. The log files with switched DD and MM time stamps are from 01/06/2019 - 08/06/2019.

For example:
In events from the 10/06/2019 log the date time is picked up correctly as 10/06/2019 (see first screenshot)
But in events from the 05/06/2019 log the date time is picket up in-correctly as 06/05/2019 (see second screenshot)

I've attached the 05/06/2019 log which is getting the incorrect date format if anyone can take a look.

How do I correct this so Splunk picks up the correct date i.e. dd/mm/yyyy?

Many thanks
Nathan

0 Karma
1 Solution

eavent_splunk
Splunk Employee
Splunk Employee

Hi Nathan,

Are you relying on automatic timestamp extraction for this ingest? If so I would recommend looking at being more explicit about telling Splunk how to interpret the date time information in your source logs - specifically, defining a TIME_FORMAT in the relevant sourcetype would be useful for you, something like this would interpret 05/06/2019 as you desire:

TIME_FORMAT = %d/%m/%Y
(Note this is for reference only - you would likely want to include hour/min/sec in your definition as well)

You can find more information on explicitly defining timestamp extraction here:
https://docs.splunk.com/Documentation/Splunk/7.3.0/Data/Configuretimestamprecognition

This info on how the timestamp processor work may be useful too:
https://docs.splunk.com/Documentation/Splunk/7.3.0/Data/HowSplunkextractstimestamps

Hope that helps!

Ed

View solution in original post

0 Karma

eavent_splunk
Splunk Employee
Splunk Employee

Hi Nathan,

Are you relying on automatic timestamp extraction for this ingest? If so I would recommend looking at being more explicit about telling Splunk how to interpret the date time information in your source logs - specifically, defining a TIME_FORMAT in the relevant sourcetype would be useful for you, something like this would interpret 05/06/2019 as you desire:

TIME_FORMAT = %d/%m/%Y
(Note this is for reference only - you would likely want to include hour/min/sec in your definition as well)

You can find more information on explicitly defining timestamp extraction here:
https://docs.splunk.com/Documentation/Splunk/7.3.0/Data/Configuretimestamprecognition

This info on how the timestamp processor work may be useful too:
https://docs.splunk.com/Documentation/Splunk/7.3.0/Data/HowSplunkextractstimestamps

Hope that helps!

Ed

0 Karma

intelli2019
New Member

Hi Ed,
Thanks for the reply.
I tried creating a new sourcetype and adding TIME_FORMAT = %d/%m/%Y. Then re-uploaded the logs, however I get the same result.
Any other ideas?
Thanks again.

0 Karma

intelli2019
New Member

It's weird because date stamps after 06/06/2019 in the new index get extracted correctly. It's only the ones on or before 06/06/2019 that get DD and MM get switched. The date format in the raw event is still in correct format.

0 Karma

eavent_splunk
Splunk Employee
Splunk Employee

I can't pretend to know the intricacies of the automatic timestamp extraction, but I do know it's not 100% reliable. I've seen this in live environments were suddenly logs stop coming in....except they are still coming in, but the day/month has been switched so the events are in the past or the future.

Anyway on your issue, if your timestamp in the log does not start at the very beginning of each event/line, you'll need to set TIME_PREFIX as well. This is a regex that should match all characters BEFORE the timestamp.

If you are able to comment a couple of lines of the working and non-working logs I might be able to take a look. Use the "code sample" formatting to make sure the original formatting is maintained, and ensure there's no sensitive data in there as well.

0 Karma

intelli2019
New Member

You sir are a legend. Adding the TIME_PREFIX worked!

0 Karma

intelli2019
New Member

Gah! seems I can't add images or files with my trial account. Hoping someone can make sense of this without them.

0 Karma
Get Updates on the Splunk Community!

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...