Getting Data In

Filter windows events on indexer from a universal forwarder

jstockamp
Communicator

I've been beating my head against a wall with this for a few hours. My setup is I have a linux indexer with a few windows universal forwarders sending data to it. I'm trying to filter out some event codes from the index. I'm assuming that since these are universal forwarders, I'll need to forward all events and filter out the ones I don't want at index time, correct?

Based on that assumption on the indexer I've got the following in $SPLUNK_HOME/etc/system/local/props.conf

[WinEventLog:Security]

TRANSFORMS-Filter_Events = FilterSecurityEvents

and in $SPLUNK_HOME/etc/system/local/transforms.conf

[FilterSecurityEvents]

REGEX=(?msi)^EventCode=(4634|4624|4769)

DEST_KEY=queue

FORMAT=nullQueue

I'm still getting events with these 3 codes in the index. Any idea(s) why this isn't working?

1 Solution

jstockamp
Communicator

I believe I've solved this. The issue is that the forwarder was running version 4.2.1 of the universal forwarder and my indexer was still running splunk version 4.2. I upgraded the indexer to 4.2 and it now seems to be working and filtering correctly.

Kind of odd that there were no errors in splunkd.log and that the forwarder clearly had no problems talking to the indexer since events were still making it into the index (just not being filtered).

View solution in original post

gwege
New Member

Any update to this? I'm having a similar issue.

0 Karma

jstockamp
Communicator

I believe I've solved this. The issue is that the forwarder was running version 4.2.1 of the universal forwarder and my indexer was still running splunk version 4.2. I upgraded the indexer to 4.2 and it now seems to be working and filtering correctly.

Kind of odd that there were no errors in splunkd.log and that the forwarder clearly had no problems talking to the indexer since events were still making it into the index (just not being filtered).

jstockamp
Communicator

Thanks. I've opened a case with support. I've tried specifying a specific host and source in props.conf instead of using a sourcetype and it still didn't work.

0 Karma

Ron_Naken
Splunk Employee
Splunk Employee

I tried your stanzas in my lab, and the events are being filtered properly. I tested with your original RegEx (which is more efficient than using no anchor, but doesn't account for possible odd multi-line behavior). You might contact support to see if there is an issue when using a forwarder. One other option might be to try specifying the transform using host or source, to see if it's limited to sourcetype.

One other consideration: My lab is eating the security eventlog as a file, rather than using WMI. The structure is exactly the same; however, the issue may have something to do with the input.

Cheers

0 Karma

jstockamp
Communicator

This is still not working. Here's my props.conf from /opt/splunk/etc/system/local:

[host::web002.xxx]
TRANSFORMS-no_lb = no_lb_traffic_transform
[WinEventLog:Security]
TRANSFORMS-FilterEvents = filtersecurityevents

and here's the transforms.conf file from the same directory:

[no_lb_traffic_transform]
REGEX=(10\.100\.0\.3|10\.100\.0\.2)(.*) 
DEST_KEY=queue
FORMAT=nullQueue
[filtersecurityevents]
REGEX=EventCode=(4634|4624|4769)
DEST_KEY=queue
FORMAT=nullQueue

As you can see, I have an additional stanza in each from some filtering I'm doing on some other web logs. That filtering is working fine, it's just the windows events that aren't being filtered. I have tested my regex in the search app and it is matching all the events that I want to filter out.

Any help would be greatly appreciated.

0 Karma

Ron_Naken
Splunk Employee
Splunk Employee

You can test your RegEx in Search, before you hit the drawing board with props/transforms. Use: ... | regex "EventCode=(4634|4624|4769)" to make sure your events show.

0 Karma

Ron_Naken
Splunk Employee
Splunk Employee

If you remove your ^ anchor, it should work:

REGEX=EventCode=(4634|4624|4769)

EDIT: Don't forget to restart Splunk after making/changing an index-time transform.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...