Quick question, is Splunk supposed to be able to understand a time stamp string like this;
2014 Mar 14 20:51:10:981 GMT -7
It seems to not understand the "-7" part. The raw data is showing up as simply GMT time.
My confusion is if altering the props.conf file will override the GMT stamp in the source data. I ~thought~ that if Splunk saw a timezone in the source data, it would take that information first over the props.conf file. I assume I'm wrong on this one and that would be a good thing.
Try using this a TIME_FORMAT in props.conf
TIME_FORMAT = %Y %b %d %H:%M:%S:%3Q %Z %z
Splunk can identify timezone by itself if its in standard format. Since your logs have custom timestamp, You need to specify TIME_FORMAT attribute to enable Splunk to identify the location of timezone in your logs. ("%Z %Z" part). You can specify TZ attribute in case the logs will miss timezone part (in that case it will take the timezone from the TZ attribute).
So, in my case, with the raw data showing
2014 Mar 14 20:51:10:981 GMT -7
I'm hosed unless I can get the user to change his logging format, correct?
As per documentation, it will use TZ from raw data first, if available. (props.conf documentation)
TZ =
* The algorithm for determining the time zone for a particular event is as follows:
* If the event has a timezone in its raw text (for example, UTC, -08:00), use that.
* If TZ is set to a valid timezone string, use that.
* If the event was forwarded, and the forwarder-indexer connection is using the
6.0+ forwarding protocol, use the timezone provided by the forwarder.
* Otherwise, use the timezone of the system that is running splunkd.
That is a non-standard timestamp. A more standard format would be "2014 Mar 14 20:51:10.981-0700". Splunk can be taught to parse your dates, however, by modifying the props.conf file. See http://answers.splunk.com/answers/4176/splunk-time-stamp-error.