Getting Data In

How to set up timestamps in this situation?

gauravnj1
Engager

Below is a sample of the log that is generated at the source. This timestamp is in UTC:

2018-02-24T21:21:43.176112 src="yy.yy.yy.yy", direction="inbound", protocol="ip", ids_type="network", vendor_product="Amun", type="amun.events", app="amun", dest="xx.xx.xx.xx", dest_port="80", signature="Connection to Honeypot", src_port="40244", sensor="xyz", transport="tcp", severity="high"

On the forwarder at this source, this is how the inputs.conf looks like:

[monitor:///var/log/mhn/mhn-splunk.log]
sourcetype = mhn
disabled = false

On the indexer, this is what I have in my props.conf:

[sourcetype=mhn]
TZ = UTC

When I search for logs in the last 24 hours, I don't get anything. When I change the timeframe to All time, that's when I see all the logs. How do I correctly set-up timestamps to make sure that I get results?

0 Karma

martin_mueller
SplunkTrust
SplunkTrust

props.conf stanzas are assumed to be sourcetypes unless prefixed with source:: or host::, so that stanza applies to a sourcetype literally called sourcetype=mhn - make sure you use just [mhn] to configure the mhn sourcetype.

The timestamp itself should be auto-recognized by Splunk, but to be sure everything works the way you want it to, I'd use these settings for timestamp recognition:

TIME_PREFIX = ^
MAX_TIMESTAMP_LOOKAHEAD = 30
TIME_FORMAT = %Y-%m-%dT%H:%M:%S.%6N
TZ = UTC

That way, there is no risk for something else in the event that might look like a timestamp to be recognized as one incorrectly. As an added bonus, helping Splunk here increases throughput when indexing.

For even more accuracy and speed, help Splunk break up events:

LINE_BREAKER = ([\r\n]+)\d\d\d\d-\d\d-\d\d
SHOULD_LINEMERGE = false
0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...