How to correct timestamp parsing and field extraction of XML log?

[user290317]

Hi, novice splunker here.

I'm having an issue in getting all the timestamps correctly parsed from the DATE and TIME fields of a given xml log.
That xml log contains exactly 68 short records of dummy client transactions. Some are parsed correctly, some incorrectly.

props.conf:

[xml_log]
TIME_PREFIX = <DATE>
TIME_FORMAT = %d%m%Y</DATE>%n<TIME>%H%M%S
SHOULD_LINEMERGE = false
LINE_BREAKER = (<\/LOG>)
REPORT-xmlext = xml-ex

It seems also that LINE_BREAKER excludes ending part of the log or '< / LOG>' since it's missing/hidden from the event listing as illustrated in screenshots above. Thank you for the kind help.

[somesoni2 · Accepted solution]

The problem with all those records where timestamp is not correct is that they occur in future. In the props.conf there is a property called MAX_DAYS_HENCE with default value of 2 and if the timestamp is more than MAX_DAYS_HENCE days from now, the timestmap will use the current timestamp. (Your timestamps are in May/December of 2017). If those dates are correct and not logging issue, then, cautiously, increase it to something larger (add to props.conf for that sourcetype only).

MAX_DAYS_HENCE = <integer>
* Specifies the maximum number of days in the future, from the current date as
  provided by input layer(For e.g. forwarder current time, or modtime for files),
  that an extracted date can be valid. Splunk still indexes events with dates
  more than MAX_DAYS_HENCE in the future with the timestamp of the last
  acceptable event. If no such acceptable event exists, new events with
  timestamps after MAX_DAYS_HENCE will use the current timestamp.
* For example, if MAX_DAYS_HENCE = 3, Splunk applies the timestamp of the last
  acceptable event to events with extracted timestamps more than 3 days in the
  future. If no acceptable event exists, Splunk applies the current timestamp.
* The default value includes dates from one day in the future.
* Defaults to 2 (days), maximum 10950.
* IMPORTANT: False positives are less likely with a tighter window, change
             with caution.

[user290317]

Thanks, it seems that it worked only after deleting event data from the index with splunk clean eventdata.
Somehow it'd always read from the old configuration of the props.conf and transforms.conf.