Getting Data In

Getting wrong timestamp in GC logs

AKG1_old1
Builder

Hello,

we have configured to pick time stamp from the logs itself but in some cases time stamp is not present. In this case Timestamp is assigned based on VM infor in GC logs.

**Logs with Timestamp**
2018-10-26T10:45:22.124+0200: 8372.847: [GC [1 CMS-initial-mark: 85682K(117800K)] 86784K(127592K), 0.0016081 secs] [Times: user=0.00 sys=0.00, real=0.00 secs] 
2018-10-26T10:47:25.532+0200: 8371.639: [GC [1 CMS-initial-mark: 65392K(117800K)] 66569K(127592K), 0.0016188 secs] [Times: user=0.00 sys=0.00, real=0.00 secs] 

**Logs without Time stamp**
69985.620: [GC [PSYoungGen: 18336K->1408K(18944K)] 31821K->14909K(45568K), 0.0322354 secs] [Times: user=0.06 sys=0.00, real=0.03 secs] 
67891.316: [GC [PSYoungGen: 17024K->928K(18944K)] 30509K->14413K(45568K), 2094.1194859 secs] [Times: user=5859.78 sys=125.11, real=2094.12 secs] 

props.conf
TIME_FORMAT = %Y-%m-%dT%H:%M:%S.%3N
TIME_PREFIX = ^

In GC logs some time VM information is printed and if there is no timestamp it pick VM time eventhough VM time is not matching with props.conf . (See attachment)
In this case eventhough logs are generated today it's displaying in year 2015. Is it possible if no timestamp present, assign upload time as time stamp ?

See attachment.alt text

0 Karma
1 Solution

FrankVl
Ultra Champion

When Splunk cannot find a timestamp, it will take the timestamp from the previous event. In your case that has a bit of an odd side effect, since the first event without a timestamp, actually does contain a timestamp, but not a useful one.

I'm not sure how you could prevent Splunk from using that "built on" timestamp, such that already that event would get assigned a timestamp based on the previous event. Perhaps reducing the MAX_TIMESTAMP_LOOKAHEAD setting in props.conf could work. Setting that to 30 should be enough to capture the proper timestamps, and may (but I'm not 100% sure of the behavior here) prevent splunk from picking up that useless build timestamp further down the event.

View solution in original post

FrankVl
Ultra Champion

When Splunk cannot find a timestamp, it will take the timestamp from the previous event. In your case that has a bit of an odd side effect, since the first event without a timestamp, actually does contain a timestamp, but not a useful one.

I'm not sure how you could prevent Splunk from using that "built on" timestamp, such that already that event would get assigned a timestamp based on the previous event. Perhaps reducing the MAX_TIMESTAMP_LOOKAHEAD setting in props.conf could work. Setting that to 30 should be enough to capture the proper timestamps, and may (but I'm not 100% sure of the behavior here) prevent splunk from picking up that useless build timestamp further down the event.

AKG1_old1
Builder

Thanks !! this worked for me.

MAX_TIMESTAMP_LOOKAHEAD=25

I tried other options as well like removing the line with VM insformation using transform but didn't worked.

Cheers!!

0 Karma
Get Updates on the Splunk Community!

Introducing the Splunk Community Dashboard Challenge!

Welcome to Splunk Community Dashboard Challenge! This is your chance to showcase your skills in creating ...

Wondering How to Build Resiliency in the Cloud?

IT leaders are choosing Splunk Cloud as an ideal cloud transformation platform to drive business resilience,  ...

Updated Data Management and AWS GDI Inventory in Splunk Observability

We’re making some changes to Data Management and Infrastructure Inventory for AWS. The Data Management page, ...