Why does SHOULD_LINEMERGE setting appear to make the date go backwards and how can I resolve?

[johnny21]

Splunk noobie here: When I configure my props.conf file like the first snippet I get multiple events showing as a single event.

[splunk@localhost ~]$ cat /opt/splunk/etc/apps/search/local/props.conf
 [sophos:syslog]
 TIME_PREFIX = <..>
 TIME_FORMAT = %Y:%m:%D-%H:%M:%S
 MAX_TIMESTAMP_LOOKAHEAD = 19

So in an attempt to resolve I modify props.conf to look like this:

[sophos:syslog]
TIME_PREFIX = <..>
TIME_FORMAT = %Y:%m:%D-%H:%M:%S
MAX_TIMESTAMP_LOOKAHEAD = 19
SHOULD_LINEMERGE = false

I get the event issue resolved BUT then my dates appear to go backward at the beginning of the new day...

Any ideas/thoughts on how to resolve?

[marycordova · Accepted solution]

You might have a typo in your TIME_FORMAT variables, specifically "D" instead of "d", you should not need line merge for these events.

[sophos:syslog]
TIME_PREFIX = <\d\d>
TIME_FORMAT = %Y:%m:%d-%H:%M:%S
MAX_TIMESTAMP_LOOKAHEAD = 19

Also, the "backwards" log events might be related to timezones. What timezone are your logs in and what timezone is your Splunk UI configured to display?

You might want to add a TZ = <timezone identifier> to props.conf also: https://answers.splunk.com/answers/127193/where-are-splunk-valid-tz-options-in-propsconf.html

[johnny21]

Thanks for the advice Mary!

So far things are looking good, I'll confirm everything works tomorrow morning once we hit a new day.

Current props.conf
[sophos:syslog]
TIME_PREFIX = <\d\d>
TIME_FORMAT = %Y:%m:%d-%H:%M:%S
MAX_TIMESTAMP_LOOKAHEAD = 19
TZ = US/Eastern

[johnny21]

Seems to have resolved my issue, many thanks!

[johnny21]

Additional information for context on what I mean by "going backwards".

[richgalloway]

You asked a similar question (https://answers.splunk.com/answers/743872/splunk-date-going-backwards.html) and accepted an answer for it. How is this different?
I see 4 events in your screen shot. All of them have the same timestamp. How is that "backwards"?
Remember, by default, Splunk displays the newest events first.

[johnny21]

Hey Rich - that is correct, I didn't want to conflate the issue as "technically" the props.conf as answered in that question addresses that one issue...it just also introduces a new one. I will try to clarify:

I am attaching a screenshot of what I mean by going backwards in the answers section below (as I can't attach screenshots in replies).

The blue line represents 11:59:59 then the red line takes us back to a previous date and inserts events from 12:00:00 and on to a previous day. The newest events are not being shown first; the newest event is at the end of the second red arrow.