Getting Data In

How can I instruct Splunk to ignore the time fields in a JSON string

abassili
Explorer

I have an input that has a JSON format:

{
"a" : 0,
"b" : 0,
"time" : 1418397877,
"timezone" : "-05:00"
}

Problem is that Splunk tries to interpret the "time" and "timzone" fields and I am getting the JSON string truncated (only the first 3 lines). How can I configure the "props.conf" file to ask Splunk to ignore those time fields?

Tags (2)
0 Karma

aljohnson_splun
Splunk Employee
Splunk Employee

DATETIME_CONFIG=NONE will prevent the timestamp extractor from running.

DATETIME_CONFIG = <filename relative to $SPLUNK_HOME>
* Specifies which file configures the timestamp extractor, which identifies timestamps from the
  event text.
* This configuration may also be set to "NONE" to prevent the timestamp extractor from running
  or "CURRENT" to assign the current system time to each event.
  * "CURRENT" will set the time of the event to the time that the event was merged from lines, or
    worded differently, the time it passed through the aggregator processor.
  * "NONE" will leave the event time set to whatever time was selected by the input layer
    * For data sent by splunk forwarders over the splunk protocol, the input layer will be the time
      that was selected on the forwarder by its input behavior (as below).
    * For file-based inputs (monitor, batch) the time chosen will be the modification timestamp on
      the file being read.
    * For other inputs, the time chosen will be the current system time when the event is read from
      the pipe/socket/etc.
  * Both "CURRENT" and "NONE" explicitly disable the per-text timestamp identification, so
    the default event boundary detection (BREAK_ONLY_BEFORE_DATE = true) is likely to not work as
    desired.  When using these settings, use SHOULD_LINEMERGE and/or the BREAK_ONLY_* , MUST_BREAK_*
    settings to control event merging.
* Defaults to /etc/datetime.xml (for example, $SPLUNK_HOME/etc/datetime.xml).

There is also MAX_TIMESTAMP_LOOKAHEAD. You could set this to a lower value so that Splunk can only look a few characters into the event for the timestamp. You may want to look at how timestamp assignment works as well.

MAX_TIMESTAMP_LOOKAHEAD = <integer>
* Specifies how far (in characters) into an event Splunk should look for a timestamp.
* This constraint to timestamp extraction is applied from the point of the TIME_PREFIX-set location.
* For example, if TIME_PREFIX positions a location 11 characters into the event, and 
  MAX_TIMESTAMP_LOOKAHEAD is set to 10, timestamp extraction will be constrained to characters 
  11 through 20.
* If set to 0, or -1, the length constraint for timestamp recognition is
  effectively disabled.  This can have negative performance implications which
  scale with the length of input lines (or with event size when LINE_BREAKER
  is redefined for event splitting).
* Defaults to 150 (characters).
0 Karma
Get Updates on the Splunk Community!

Announcing Scheduled Export GA for Dashboard Studio

We're excited to announce the general availability of Scheduled Export for Dashboard Studio. Starting in ...

Extending Observability Content to Splunk Cloud

Watch Now!   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to leverage ...

More Control Over Your Monitoring Costs with Archived Metrics GA in US-AWS!

What if there was a way you could keep all the metrics data you need while saving on storage costs?This is now ...