The heavy forwarders we are leveraging appear to be ignoring the props.conf file, resulting in timestamp extraction issue. This particular input is through HEC.
I managed to use "splunk btool props list" to verify that the props values are loaded to memory, but I do not see any evidence that it's being used.
Sourcetype headings match.
Thanks in advance!
Below answer may help:
https://answers.splunk.com/answers/716770/defining-timestamp-for-hec-input.html
What are your configurations for the HEC endpoint? Looking for the default sourcetype for inbound data.
Also, can you provide a sanitized sample of how you're invoking the HEC endpoint, what parameters you are using in your testing?
in props.conf
[<sourcetypename>]
disabled=false
SHOULD_LINEMERGE=false
LINE_BREAKER=([\r\n]+)\{\"\w+\"\:
TIME_FORMAT=%Y-%m-%d %H:%M:%S.%3N
TIME_PREFIX=\"message\"\:\"
MAX_TIMESTAMP_LOOKAHEAD=50
TRUNCATE=10000
[http://<token_name>]
disabled = 0
token = ****
index = <index_name>
indexes = <index_name>
queueSize = 4MB
useACK = false
. . .
<appender name="splunkAppender" class="com.splunk.logging.HttpEventCollectorLogbackAppender">
<url>${DATACORE_SPLUNK_URL}</url>
<token>${DATACORE_SPLUNK_TOKEN}</token>
<source>datacore-dal-${SPRING_PROFILES_ACTIVE}</source>
<sourcetype><sourcetypename></sourcetype>
<disableCertificateValidation>true</disableCertificateValidation>
<layout class="ch.qos.logback.classic.PatternLayout">
<pattern>${LOG_PATTERN}</pattern>
</layout>
</appender>
. . .
Message content
{"severity":"WARN","message":"2019-01-19 22:26:47.604 . . . }