Getting Data In

Splunk randomly extracts 2 types of timestamp formats!

Dark_Ichigo
Builder

I have no idea what I missing here, just no idea and I have to admit, its killing me inside, I have been stuck on this for 2 weeks!

for some random reason, Splunk decides to index all my timestamps in Australian Format (Which is what I want!), but decides to index a small number of them in American format (even though they're from the same Log!).

Here is a copy of the sourcetype stanza in props.conf:

TIME_FORMAT = %d/%m/%Y %H:%M:%S.%3N
TZ = Australia/Victoria
TIME_PREFIX = ^

And here is a copy of log Im ingesting:

What Splunk Gets: 05/01/2013 11:19:37.222

What the log really states: [01/05/2013 11:19:37.222 INFO ] - [AuditLogger] - SessionId=#####; UserId=#####; Event=#####; MSISDN=#######

And please note, it only does this for a small number of events like the above, the other timestamps are extracted in the correct format!, all the other events look exactly like the one I pasted above, so I have no idea WHAT TO DO NEXT!

Please all I want is for my logs to be indexed in Australian format, Plz

0 Karma
1 Solution

kristian_kolb
Ultra Champion

The TIME_PREFIX should/must also include the opening square bracket. I think that until now, your Splunk has been able to deduce from numbers alone that e.g. 25/4/2013 cannot be in %m/%d/%Y format.

TIME_PREFIX = ^\[

Hope this helps,

Kristian

View solution in original post

kristian_kolb
Ultra Champion

The TIME_PREFIX should/must also include the opening square bracket. I think that until now, your Splunk has been able to deduce from numbers alone that e.g. 25/4/2013 cannot be in %m/%d/%Y format.

TIME_PREFIX = ^\[

Hope this helps,

Kristian

Dark_Ichigo
Builder

Thanks, that was one of the issues, the other one was that someone had names the stanza in props.conf to the index name instead of the Sourcetype name, now its fixed 🙂

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...