Getting Data In

props.conf timestamp clarification

wwhite12
Path Finder

I have json data that can vary greatly in size with the timestamp field coming at the end of each event. I'm able to parse all the timestamps correctly using the config TIME_PREFIX="timestamp":+ except for the events that are very large. My question is, in order to parse the timestamp for the very large events, do I need to add a MAX_TIMESTAMP_LOOKAHEAD? Or if I added a larger TRUNCATE would the TIME_PREFIX config still need the MAX_TIMESTAMP_LOOKAHEAD?

props.conf
[mysourcetype]
CHARSET=UTF-8
INDEXED_EXTRACTIONS=json
KV_MODE=none
LINE_BREAKER=([\r\n]+)
NO_BINARY_CHECK=true
SHOULD_LINEMERGE=true
category=Structured
description=JavaScript Object Notation format. For more information, visit http://json.org/
disabled=false
pulldown_type=true
TIME_PREFIX="timestamp":+

0 Karma
1 Solution

richgalloway
SplunkTrust
SplunkTrust

The MAX_TIMESTAMP_LOOKAHEAD settings starts at TIME_PREFIX so changing it won't help. It's likely you're running into your TRUNCATE limit. Try increasing that after you make sure events are breaking correctly.

---
If this reply helps you, Karma would be appreciated.

View solution in original post

shivanshu1593
Builder

As @richgalloway rightly pointed, you should look into increasing the value of TRUNCATE (Defaults to 10,000). Splunk logs it's complain regarding the truncate issues in splunkd.log inside $SPLUNK_HOME/var/log/splunk. You can check it, to make sure you're facing the same issue.

Thank you,
Shiv
###If you found the answer helpful, kindly consider upvoting/accepting it as the answer as it helps other Splunkers find the solutions to similar issues###
0 Karma

richgalloway
SplunkTrust
SplunkTrust

The MAX_TIMESTAMP_LOOKAHEAD settings starts at TIME_PREFIX so changing it won't help. It's likely you're running into your TRUNCATE limit. Try increasing that after you make sure events are breaking correctly.

---
If this reply helps you, Karma would be appreciated.
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...