Getting Data In

How to set up timestamps in this situation?

gauravnj1
Engager

Below is a sample of the log that is generated at the source. This timestamp is in UTC:

2018-02-24T21:21:43.176112 src="yy.yy.yy.yy", direction="inbound", protocol="ip", ids_type="network", vendor_product="Amun", type="amun.events", app="amun", dest="xx.xx.xx.xx", dest_port="80", signature="Connection to Honeypot", src_port="40244", sensor="xyz", transport="tcp", severity="high"

On the forwarder at this source, this is how the inputs.conf looks like:

[monitor:///var/log/mhn/mhn-splunk.log]
sourcetype = mhn
disabled = false

On the indexer, this is what I have in my props.conf:

[sourcetype=mhn]
TZ = UTC

When I search for logs in the last 24 hours, I don't get anything. When I change the timeframe to All time, that's when I see all the logs. How do I correctly set-up timestamps to make sure that I get results?

0 Karma

martin_mueller
SplunkTrust
SplunkTrust

props.conf stanzas are assumed to be sourcetypes unless prefixed with source:: or host::, so that stanza applies to a sourcetype literally called sourcetype=mhn - make sure you use just [mhn] to configure the mhn sourcetype.

The timestamp itself should be auto-recognized by Splunk, but to be sure everything works the way you want it to, I'd use these settings for timestamp recognition:

TIME_PREFIX = ^
MAX_TIMESTAMP_LOOKAHEAD = 30
TIME_FORMAT = %Y-%m-%dT%H:%M:%S.%6N
TZ = UTC

That way, there is no risk for something else in the event that might look like a timestamp to be recognized as one incorrectly. As an added bonus, helping Splunk here increases throughput when indexing.

For even more accuracy and speed, help Splunk break up events:

LINE_BREAKER = ([\r\n]+)\d\d\d\d-\d\d-\d\d
SHOULD_LINEMERGE = false
0 Karma
Get Updates on the Splunk Community!

Introducing the 2024 SplunkTrust!

Hello, Splunk Community! We are beyond thrilled to announce our newest group of SplunkTrust members!  The ...

Introducing the 2024 Splunk MVPs!

We are excited to announce the 2024 cohort of the Splunk MVP program. Splunk MVPs are passionate members of ...

Splunk Custom Visualizations App End of Life

The Splunk Custom Visualizations apps End of Life for SimpleXML will reach end of support on Dec 21, 2024, ...