Getting Data In

Can we assign a timestamp to the previous line while uploading it to Splunk?

AKG1_old1
Builder

Hi,

In a raw file, some of the lines don't have a timestamp at the start of the line. can we assign timestamp of the previous line while uploading it to Splunk ?

In the data below 3rd row, doesn't have a timestamp. Is it possible to add "2017-05-16T13:21:47.342+0200:" at the beginning of the 3rd line?

- 2017-05-16T13:21:46.592+0200: [GC428241.647: [ParNew (promotion failed): 411872K->419456K(419456K), 0.2304607 secs]428241.877: [CMS428254.896: [CMS-concurrent-sweep: 13.949/15.064 secs] [Times: user=36.16 sys=0.32, real=15.06 secs] 
- 2017-05-16T13:21:47.342+0200: [ParNew (promotion failed): 396564K->419456K(419456K), 0.2080311 secs]427705.734: [CMS427713.980: [CMS-concurrent-sweep: 12.295/15.565 secs] [Times: user=115.61 sys=1.24, real=15.57 secs] 
- (concurrent mode failure): 24567602K->12581726K(24641536K), 35.9159001 secs] 24935117K->12581726K(25060992K), [CMS Perm : 79323K->78757K(262144K)], 36.1246617 secs] [Times: user=36.62 sys=0.04, real=36.12 secs] 

Thanks

0 Karma

niketn
Legend

@agoyal, Similar to the attached screenshot, can you try to upload a dummy log file with the three events as per your question?
If all your events break with secs] the add the same as BREAK_ONLY_BEFORE criteria. While the Timestamp recognition settings are not mandatory, you can add the following to tell Splunk specifically where to look for timestamp and what format the timestamp is going to be.

Once you do these settings you will have three events with the third event's timestamp same as second event's timestamp.

[yourSourceType]
SHOULD_LINEMERGE=true
NO_BINARY_CHECK=true
BREAK_ONLY_BEFORE=secs]
TIME_FORMAT=%Y-%m-%dT%H:%M:%S.%3N%z
TIME_PREFIX=\s-\s
MAX_TIMESTAMP_LOOKAHEAD=28

alt text
Please try out and confirm.

____________________________________________
| makeresults | eval message= "Happy Splunking!!!"

AKG1_old1
Builder

Hi @niketnilay, I tried above soultion and its working if there is proper time stamp all other line. but unfortunately it wont work for me. In our log files sometimes instead of time stamp it have some numbers(its relative_time which is used to evaluate proper time).

example:
546385.203: [GC546385.203: ParNew: 327126K->18279K(419456K), 0.0268272 secs: 24567602K->12581726K(24641536K)

is it possible to add this number to next line which doen't have this format.

Thanks for help
Ankit

0 Karma

niketn
Legend

@agoyal, if this worked for you please accept the answer to mark this as answered.

____________________________________________
| makeresults | eval message= "Happy Splunking!!!"
0 Karma

niketn
Legend

@agoyal what does timestamp like 427705.525 resolve to? Does not seem like epoch time.
If Splunk is able to reolve the timestamp for first two events and not for the third event as part of its default timestamp recognition algorithm it should automatically set the time for third event same as the second event.

http://docs.splunk.com/Documentation/Splunk/latest/Data/HowSplunkextractstimestamps

Refer to the following behavior with Timestamp in %Y/%m/%d %H:%M:%S.%3N format.

alt text

____________________________________________
| makeresults | eval message= "Happy Splunking!!!"

AKG1_old1
Builder

for better understanding I have updated raw data. I have replaced the relative_time with timestamp.

0 Karma

AKG1_old1
Builder

Hi, It's bit tricky. It's Actually relative_time, so getting the epoch time I have to add start_time to this epoch time.
Start_time is captured from other log file.

basically I want to add this relative_time from previous or next line to the lines which doen't have it.

0 Karma

niketn
Legend

Are you doing this event timestamp identification during index time or search time? Are you using datetime.xml by any chance?

____________________________________________
| makeresults | eval message= "Happy Splunking!!!"
0 Karma

AKG1_old1
Builder

relative_time(427705.525) identification is done on index time. And in search query we add start_time+relative_time to get effective _time. I am not using datetime.xml

0 Karma
Get Updates on the Splunk Community!

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...