Below is a sample of the log that is generated at the source. This timestamp is in UTC:
2018-02-24T21:21:43.176112 src="yy.yy.yy.yy", direction="inbound", protocol="ip", ids_type="network", vendor_product="Amun", type="amun.events", app="amun", dest="xx.xx.xx.xx", dest_port="80", signature="Connection to Honeypot", src_port="40244", sensor="xyz", transport="tcp", severity="high"
On the forwarder at this source, this is how the inputs.conf looks like:
[monitor:///var/log/mhn/mhn-splunk.log]
sourcetype = mhn
disabled = false
On the indexer, this is what I have in my props.conf:
[sourcetype=mhn]
TZ = UTC
When I search for logs in the last 24 hours, I don't get anything. When I change the timeframe to All time, that's when I see all the logs. How do I correctly set-up timestamps to make sure that I get results?
props.conf stanzas are assumed to be sourcetypes unless prefixed with source::
or host::
, so that stanza applies to a sourcetype literally called sourcetype=mhn
- make sure you use just [mhn]
to configure the mhn
sourcetype.
The timestamp itself should be auto-recognized by Splunk, but to be sure everything works the way you want it to, I'd use these settings for timestamp recognition:
TIME_PREFIX = ^
MAX_TIMESTAMP_LOOKAHEAD = 30
TIME_FORMAT = %Y-%m-%dT%H:%M:%S.%6N
TZ = UTC
That way, there is no risk for something else in the event that might look like a timestamp to be recognized as one incorrectly. As an added bonus, helping Splunk here increases throughput when indexing.
For even more accuracy and speed, help Splunk break up events:
LINE_BREAKER = ([\r\n]+)\d\d\d\d-\d\d-\d\d
SHOULD_LINEMERGE = false