Splunk Heavy Forwarder Filtering and Windows Universal Forwarders

[yongly]

Hi so the set up we have is universal forwarders -> heavy forwarder -> Indexer..

Where the heavy forwarder acts as an intermediate filtering server. The idea that all data that gets sent to the heavy forwarder gets dropped unless otherwise specified. To do this, I have a default stanza in props.conf.

This seems to work fine from what I can see when tested with Universal Forwarders forwarding data from unix. But for some reason, when I added installed a universal forwarder on a windows machine I expected the events to be dropped until I added a specific stanza for that machine.. but for some reason it seems to by pass this and it gets indexed on the Splunk Indexer..

Does anyone have an idea as to why this might be the case??

Configuration included below:

inputs.conf:

[default]

host = snzclakl128

[splunktcp://9996]

Props.conf:

[default]

TRANSFORMS-routing=discard_all

// Filter out everything except...

[source::/home/app/content/logs/ssl_access_log]

TRANSFORMS-routing=discard_all,ssl_access_log_allow

// Filter out everything except...

[source::/home/app/httpd/logs/ssl_request_log]

TRANSFORMS-routing=discard_all,ssl_request_log_allow

// Discard all DEBUG entries and allow the rest

[source::/opt/zenoss/log/localhost/zeneventlog.log]

TRANSFORMS-routing=allow_all,zeneventlog_discard

Transforms.conf

// Use this transform to discard all entries from log file. This is the default

[discard_all]

REGEX=.

DEST_KEY=queue

FORMAT=nullQueue

// Use this transform to allow and forward all entries from log file to indexer
[allow_all]

REGEX=.

DEST_KEY=queue

FORMAT=indexQueue

[Ayn]

The problem is likely that you don't have unique identifiers for your TRANSFORMS definitions. All of them are called "TRANSFORMS-routing". These identifiers need to be unique, because that's how Splunk keeps track of them. So, if you have two with the same name, the second one is "overwriting" the first. So if you call them something like "TRANSFORMS-routing_ssl_access_log_allow", "TRANSFORMS-routing_discard_all" and so on you might solve your problem. (or at least we can go on to the next step of troubleshooting 😉 )

[yongly]

Those identifiers are linked to the stanzas which define which transform to use so it shouldn't really be a problem. Nevertheless I did try it and it didn't work. any other ideas? 🙂

[yongly]

Further testing showed that an addition to props.conf for that specific sourcetype gets ignored as well. I wonder if it's going to a different queue or something?

[WMI:WinEventLog:Security]
TRANSFORMS-routing=discard_all