Hi.
I'm trying to find a quick and simple way to separate my incoming cisco syslogs into different indexes. For complicated, and dull reasons we can only really use the udp:514 listener.
The best I've come up with so far is defining a number of different transforms in the props.conf, then use regex to define which index they go to. unfortunately my splunk and regex skills are currently rather weak and it's refusing to work.
So far I've got to here:
~~ props.conf ~~
[syslog]
TRANSFORMS-routing = generic-routing, MPLS-routing, Office-routing
~~ transforms.conf ~~
[MPLS-routing]
SOURCE_KEY = MetaData:Host
REGEX = 10\.10\.\d+
DEST_KEY = _MetaData:Index
FORMAT = MPLS
[Office-routing]
SOURCE_KEY = MetaData:Host
REGEX = 10\.100\.\d+
DEST_KEY = _MetaData:Index
FORMAT = office
[generic-routing]
DEST_KEY = _MetaData:Index
FORMAT = syslog
Any tips or thoughts?
apologies for not getting back to you, but I eventually got this working in a similar method to above.
I ended up going for a [source::udp:514] thingy in props.conf then regex'ing in transforms as we discussed.
the trick was, as usual, getting the regex to work. ended up with the regex below to match the basic date/timestamp format of "MMM DD HH:MM:SS I{P OR Hostname}". pretty simple stuff but invaluable nonetheless.
[office_routing]
REGEX = (\w+\ [0-9]{1,2}\ \d+:\d+:\d+ 10.100.[0-9]{1,3}.[0-9]{1,3})|(\w+.domain.ltd)
DEST_KEY = _MetaData:Index
FORMAT = office
And it works perfectly
thanks.
One thing that could mess things up, and possibly be the source of your problems is that incoming syslog (i.e. with the sourcetype
set to syslog
) data goes through a TRANSFORM
for setting the host value based on the content of the events. Have a look at the $SPLUNK_HOME/etc/system/default/props.conf
and transforms.conf
to see how this works.
If this happens before your TRANSFORMS
the host would not be there, or rather may not have the values you expect.
You can skip the SOURCE_KEY
setting (default is the _raw event) and create your regexes to match a timestamp followed by 10.10.*
or 10.100.*
etc.
Also, you may need to rename the the transforms so they use underscores instead of hyphens;
generic_routing
MPLS_routing
Office_routing
Also, you may have to specify a REGEX = .
for the [generic_routing]
. (see new spelling)
Hope this helps,
Kristian
Try the bottom two things first. (REGEX = . and underscores)
/K
Would it make more sense to use [source::udp] in the props.conf instead of [syslog] ?