how to specify props.conf for EPOCH timestamp in JSON object

[ronak]

I've events coming in JSON format with first part of the JSON data as EPOCH_START_TIME=8797994058574 ...the events are sent on Universal forwarder on some port that absorbs these events.

Question

1. How do I specify the props.conf to recognize this to be the timestamp for the event (so that Splunk does not add index time as timestamp)

[martin_mueller]

Do post an entire event along with the current configuration you use for that sourcetype.
While Splunk will recognize epoch timestamps out of the box with zero configuration, some defaults may trip you. For example, it'll only look at the first 150 characters of each event by default.

[woodcock]

The problem is that your timestamp is epoch but it is in milliseconds, not seconds, which is why it looks funny and Splunk may not understand. It could not the last time I tried it (which was a long time ago) so I pre-processed it by dividing by 1000.

[dshpritz]

Splunk does support it, and has since at least v4. See my answer. If you specify your TIME_FORMAT to include them, then you will get them, using %s (time in seconds) + %

[martin_mueller]

Splunk has no problem whatsoever recognizing millisecond-precision epoch timestamps.

This was indexed using a non-existing sourcetype, ie using Splunk's default settings:

Additionally, I doubt the question contains a millisecond-precision epoch timestamp - the example given would map to the year 2248. Not impossible, but quite unlikely indeed.

[ronak]

Martin - quite correct interpretation...EPOCH_START_TIME= 1434209641 would be actual the value. (sample data)

Still, Splunk did not recognize. When I removed the TIME_PREFIX from props.conf, it did pick it up for one of the inputs.

Very strange behavior.

[dshpritz]

Here are some props.conf settings. These would go in a local/props.conf on your indexers or heavy forwarders:

[mysourcetype]
TIME_PREFIX = EPOCH_START_TIME=
TIME_FORMAT = %s%4N
MAX_TIMESTAMP_LOOKAHEAD = 14

Thanks,

Dave

[martin_mueller]

This would map the example timestamp to 1997 - if that's the correct interpretation you will also need to set this for mysourcetype:

MAX_DAYS_AGO = 10000

By default, Splunk will not accept timestamps older than 2000 days and instead considers them a misconfigured timestamp recognition.

[martin_mueller]

Are you sure that's an epoch timestamp? I'd expect ten digits starting with a one for second precision, or thirteen digits also starting with a one for millisecond precision.

https://en.wikipedia.org/wiki/Epoch_time

[ronak]

EPOCH_START_TIME=1434209641