Getting Data In

Time Stamps 1 day early?

empath
Explorer

I've got a log file which tracks some call statistics.

For some reason, about half of these, Splunk has them as being exactly 1 day older than they are.

This one is indexed as 7/15/10

VQM: 8885551212 07/16/10.16:59:59 0.000% 0ms - G726 20ms 1 1

This one is indexed as 07/16/10

VQM: 8885551212 07/16/10.16:33:40 0.000% 0ms - G729 20ms 1 1

For the life of me, I cannot tell the difference between the two.

I found a field called timestartpos, but that just (correctly) shows the first character of the time. Is there something like that for date?

I also correctly set the Time Zone in props.conf and created a timeformat string for that source, which didn't make any difference.

Thanks for your assistance.

Tags (1)
1 Solution

Lowell
Super Champion

I'm pretty sure timestartpos and timeendpos show the start/end position of the entire timestamp not just the time portion of it. (Although, I don't remember every verifying that, so it could just be an assumption.)

My guess would be that splunk is detecting some part of your date as a different timezone. You can look at the field date_zone and see if it is different between your indexed events.

As far as your props.conf entries, just to confirm you've tried something like this:

 [my_source_type]
 TIME_PREFIX = ^\S+ \d+
 TIME_FORMAT = %m/%d/%y.%H:%M:%S
 SHOULD_LINEMERGE = False

BTW, I'm not sure about the output of splunk date train stuff, the formatting is hard to follow in your comment, and to be honest, I've never found that tool all that useful. Setting an explicit TIME_FORMAT has always been easier and less awkward IMHO.

View solution in original post

Lowell
Super Champion

I'm pretty sure timestartpos and timeendpos show the start/end position of the entire timestamp not just the time portion of it. (Although, I don't remember every verifying that, so it could just be an assumption.)

My guess would be that splunk is detecting some part of your date as a different timezone. You can look at the field date_zone and see if it is different between your indexed events.

As far as your props.conf entries, just to confirm you've tried something like this:

 [my_source_type]
 TIME_PREFIX = ^\S+ \d+
 TIME_FORMAT = %m/%d/%y.%H:%M:%S
 SHOULD_LINEMERGE = False

BTW, I'm not sure about the output of splunk date train stuff, the formatting is hard to follow in your comment, and to be honest, I've never found that tool all that useful. Setting an explicit TIME_FORMAT has always been easier and less awkward IMHO.

empath
Explorer

Thanks, I had done Time_format, but I hadn't added the time prefix line! thanks.

0 Karma

empath
Explorer

So my original questions is somewhat misleading. My question is now: Why isn't Splunk recognizing the date?

0 Karma

empath
Explorer

Additional notes:

I ran splunk train date on this data, and for some reaosn it can't seem to find the date:

From: VQM: 8885551212 07/16/10.16:03:59 0.000% 0ms - G726 20ms 1 1
Parsed: Fri Jul 16 16:03:59 2010
UTC Time: 1279310639
Time Region: 25-34
Date Region: -1--1
From: VQM: 8885551212 07/16/10.16:03:59 0.000% 0ms - G726 20ms 1 1
Parsed: Fri Jul 16 16:03:59 2010
UTC Time: 1279310639
Time Region: 25-34
Date Region: -1--1

I assume Date Region 1 1 means that it couldn't find it?

0 Karma
Get Updates on the Splunk Community!

Introducing the 2024 SplunkTrust!

Hello, Splunk Community! We are beyond thrilled to announce our newest group of SplunkTrust members!  The ...

Introducing the 2024 Splunk MVPs!

We are excited to announce the 2024 cohort of the Splunk MVP program. Splunk MVPs are passionate members of ...

Splunk Custom Visualizations App End of Life

The Splunk Custom Visualizations apps End of Life for SimpleXML will reach end of support on Dec 21, 2024, ...