Getting Data In

How can I instruct Splunk to ignore the time fields in a JSON string

abassili
Explorer

I have an input that has a JSON format:

{
"a" : 0,
"b" : 0,
"time" : 1418397877,
"timezone" : "-05:00"
}

Problem is that Splunk tries to interpret the "time" and "timzone" fields and I am getting the JSON string truncated (only the first 3 lines). How can I configure the "props.conf" file to ask Splunk to ignore those time fields?

Tags (2)
0 Karma

aljohnson_splun
Splunk Employee
Splunk Employee

DATETIME_CONFIG=NONE will prevent the timestamp extractor from running.

DATETIME_CONFIG = <filename relative to $SPLUNK_HOME>
* Specifies which file configures the timestamp extractor, which identifies timestamps from the
  event text.
* This configuration may also be set to "NONE" to prevent the timestamp extractor from running
  or "CURRENT" to assign the current system time to each event.
  * "CURRENT" will set the time of the event to the time that the event was merged from lines, or
    worded differently, the time it passed through the aggregator processor.
  * "NONE" will leave the event time set to whatever time was selected by the input layer
    * For data sent by splunk forwarders over the splunk protocol, the input layer will be the time
      that was selected on the forwarder by its input behavior (as below).
    * For file-based inputs (monitor, batch) the time chosen will be the modification timestamp on
      the file being read.
    * For other inputs, the time chosen will be the current system time when the event is read from
      the pipe/socket/etc.
  * Both "CURRENT" and "NONE" explicitly disable the per-text timestamp identification, so
    the default event boundary detection (BREAK_ONLY_BEFORE_DATE = true) is likely to not work as
    desired.  When using these settings, use SHOULD_LINEMERGE and/or the BREAK_ONLY_* , MUST_BREAK_*
    settings to control event merging.
* Defaults to /etc/datetime.xml (for example, $SPLUNK_HOME/etc/datetime.xml).

There is also MAX_TIMESTAMP_LOOKAHEAD. You could set this to a lower value so that Splunk can only look a few characters into the event for the timestamp. You may want to look at how timestamp assignment works as well.

MAX_TIMESTAMP_LOOKAHEAD = <integer>
* Specifies how far (in characters) into an event Splunk should look for a timestamp.
* This constraint to timestamp extraction is applied from the point of the TIME_PREFIX-set location.
* For example, if TIME_PREFIX positions a location 11 characters into the event, and 
  MAX_TIMESTAMP_LOOKAHEAD is set to 10, timestamp extraction will be constrained to characters 
  11 through 20.
* If set to 0, or -1, the length constraint for timestamp recognition is
  effectively disabled.  This can have negative performance implications which
  scale with the length of input lines (or with event size when LINE_BREAKER
  is redefined for event splitting).
* Defaults to 150 (characters).
0 Karma
Get Updates on the Splunk Community!

Introducing the Splunk Community Dashboard Challenge!

Welcome to Splunk Community Dashboard Challenge! This is your chance to showcase your skills in creating ...

Wondering How to Build Resiliency in the Cloud?

IT leaders are choosing Splunk Cloud as an ideal cloud transformation platform to drive business resilience,  ...

Updated Data Management and AWS GDI Inventory in Splunk Observability

We’re making some changes to Data Management and Infrastructure Inventory for AWS. The Data Management page, ...