Getting Data In

Parsing Timestamps from Windows and Cisco devices from Props.conf

MikeyG
Explorer

I have one Sourcetype [syslog-ng] with a myriad of devices all sending (IDS; CISCO; Linux; Windows Servers) through it. When I added the Cisco devices discovered that %FWSM events do not have a timestamp and indexing slowed WAY down .. apparently Splunk is parsing everything looking for a timestamp. Given the following which parses windows events correctly, skips the first timestamp, and retrieves the correct timestamp:

[syslog]
MAX_TIMESTAMP_LOOKAHEAD =128
TIME_PREFIX = ^[A-Z][a-z]+\s+\d+\s\d+:\d+:\d+\s[^\s]*\s
TRANSFORMS-strip-syslog = syslog-header-stripper-ts-host
TRANSFORMS-ip_address = basic_ip_address
TRANSFORMS-win_snare = windows-snare
REPORT-snare = windows-snare, windows-snare-group-mod
TRANSFORMS-sfire_ids = sourcefire
REPORT-ids_evnts = sourcefire
TRANSFORMS-nms_data = NMS

Can I add the following to grab the first timestamp in the event?

# Cisco Firewall Service Modules - Where %FWSM messages do not contain a timestamp
[host :: XXX-FW-*]
MAX_TIMESTAMP_LOOKAHEAD = 32
TIME_FORMAT = %b %d %H:%M:%S
0 Karma
1 Solution

gkanapathy
Splunk Employee
Splunk Employee

Possibly. When [host::myhostname] and [mysourcetype] settings both apply, the settings will all be merged, and in case of conflict, the [host::myhostname] settings win. You may also need to modify the TIME_PREFIX as Splunk will not look for a timestamp until after that match.

However, note that [host::myhostname] settings will only apply if you are not relying on a tranformed host name. If your data is coming in to Splunk via UDP, then host value will be the IP address from which it was received. If it is coming in via a Splunk forwarder, it will be whatever the forwarder reports. It will not be the value extracted by syslog-header-stripper-ts-host, since the host has not been set to that value at the time the rules are selected.

Another alternative is to create a custom datetime.xml file with the two formats listed in appropriate priority.

View solution in original post

0 Karma

rforsythe
Engager

Also make sure you have 'logging timestamp' enabled on your FWSM module (and other Cisco devices) to include that in messages. Cisco logging is very flexible, but that means the default config isn't always how you'd like it.

gkanapathy
Splunk Employee
Splunk Employee

Possibly. When [host::myhostname] and [mysourcetype] settings both apply, the settings will all be merged, and in case of conflict, the [host::myhostname] settings win. You may also need to modify the TIME_PREFIX as Splunk will not look for a timestamp until after that match.

However, note that [host::myhostname] settings will only apply if you are not relying on a tranformed host name. If your data is coming in to Splunk via UDP, then host value will be the IP address from which it was received. If it is coming in via a Splunk forwarder, it will be whatever the forwarder reports. It will not be the value extracted by syslog-header-stripper-ts-host, since the host has not been set to that value at the time the rules are selected.

Another alternative is to create a custom datetime.xml file with the two formats listed in appropriate priority.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...