I have all my switch and linux syslogs stored in a single directory - let's call it /var/log/syslog. I'm trying to assign specific indexes and sourcetypes based on a regex statement of the filename. Here's what I have:
[monitor:///var/log/syslog]
whitelist = \/var\/log\/syslog\/^sw.*
disabled = false
index = switches
host_segment = 4
sourcetype = switch_syslog
[monitor:///var/log/syslog]
whitelist = \/var\/log\/syslog\/^[^s][^w].*
disabled = false
index = linux
host_segment = 4
sourcetype = linux_syslog
These are two monitoring stanzas of the same directory. One is pulling out filenames that begin with "sw", and the other is pulling out filenames beginning with anything but "sw". I'm able to capture some of the logs, but not all. Can you see a better way to achieve the same thing?
Thank you.
Ed
Hi there, I haven't tested it but this migth guide you.
Create a single input between those two.
inputs.conf
[monitor:///var/log/syslog]
whitelist = (\/var\/log\/syslog\/^sw.*|\/var\/log\/syslog\/^[^s][^w].*)
disabled = false
host_segment = 4
sourcetype = syslog_routing
props.conf
[syslog_routing]
TRANSFORMS-convert_to_sourcetypes = convert_to_switch_syslog, convert_to_linux_syslog
TRANSFORMS-route_by_index = move_to_index_switch, move_to_index_linux
transforms.conf
[convert_to_switch_syslog]
SOURCE_KEY = MetaData:Source
REGEX = ^source::\/var\/log\/syslog\/^sw.*
DEST_KEY = MetaData:Sourcetype
FORMAT = sourcetype::switch_syslog
[convert_to_linux_syslog]
SOURCE_KEY = MetaData:Source
REGEX = ^source::\/var\/log\/syslog\/^[^s][^w].*
DEST_KEY = MetaData:Sourcetype
FORMAT = sourcetype::linux_syslog
[move_to_index_switch]
SOURCE_KEY = MetaData:Sourcetype
DEST_KEY = _MetaData:Index
REGEX = switch_syslog
FORMAT = switches
[move_to_index_linux]
SOURCE_KEY = MetaData:Sourcetype
DEST_KEY = _MetaData:Index
REGEX = linux_syslog
FORMAT = linux
Splunk will be applying monitoring based on only one monitoring stanza as it merges all configurations available on the forwarder (all splunk instances, just using forwarder as an example), probably the last one. You can just have one monitoring stanza with combined whitelist (merge regex) and then use props-transforms on the indexer/heavy forwarder to override the metadata fields. See this for later part.
https://answers.splunk.com/answers/301504/how-to-override-sourcetype-and-index-assignment.html