Why is the blacklist filter in inputs.conf on a universal forwarder not filtering out EventCode 5156 as expected?

[noybin]

Hi,

I have a Splunk 6.2.5 Universal Forwarder 6.2.5 on a Windows 7 forwarding logs to Splunk 6.2.5 on a Linux server.

I've configured the C:\Program Files\SplunkUniversalForwarder\etc\system\local\inputs.conf to filter events with EventCode 5156:

[default]
host = PROGRESSIVE
[WinEventLog://Application]
disabled = 0
[WinEventLog://Security]
blacklist= 5156
disabled = 0
[WinEventLog://System]
disabled = 0
[WinEventLog://Setup]
disabled = 1
[WinEventLog://Windows-System32-winevt-Logs/Microsoft-Windows-Sysmon%4Operational]
disabled = 0
[script://$SPLUNK_HOME\bin\scripts\splunk-wmi.path]
disabled = 0

The problem is that those events are not being filtered:

I've restarted the UF and the Splunk Server as well without results.

Thank you very much.

[mali8]

Hello,

I have the same problem :S
noybin did you find the fix to filter events (5156)?

I treid with blacklist, whitelist and the events are still indexed :S

props.conf and transforms.conf didn't fix the problem for me :S

Splunk Version: 6.2.1
Splunk Build: 245427

[noybin]

Hi,

I've solved the problem by uninstalling the Uinversal Forwarder and reinstalling it.
But this time when selecting what to monitor I selected each of the event types to monitor instead of selecting the windows events log directory as I did the previous time.

That solved the issue.

[tskinnerivsec]

Whitelist/blacklist operates on the file level, i.e. which files to include/exclude for indexing. If you are looking to black list particular event ids out of a log file, you need to define a transform in a transforms.conf file and call it with a props.conf file on a heavy forwarder or indexer that your event will pass through.

for example:

props.conf -

[WinEventLog://Security]
TRANSFORMS-send_to_nullqueue = send_to_nullqueue

transforms.conf

[send_to_nullqueue]
DEST_KEY = queue
REGEX = EventCode=5156
FORMAT = nullQueue

These can be deployed via their own configuration app, or include them in the /local of the Splunk_TA_windows if you are using that.

[noybin]

I've also tried:

[WinEventLog://Security]
blacklist1=EventCode="5156"
disabled = 0

For more than an hour it stopped showing events with eventCode=5156 although I could see them on the eventViewer . But Afterwards it started again finding these events.

Now I copied that 3 lines into the inputs.conf in C:\Program Files\SplunkUniversalForwarder\etc\apps\Splunk_TA_windows\local

By the time Splunk is still finding events (5156)

[noybin]

Updating:
I find a strange behaviour with this events: While other events are continuously being logeed in Splunk, the 5156 events are logged sometimes. I can be an hour without finding any 5156 events and then I find 10.000 of 5156 events that come all together.

If I configure the props.conf and transforms.conf to filter these events as told above, are they still being indexed? Are they going to count on the license cuota?

Thank you.

[tskinnerivsec]

No, they would get filtered out before being indexed.

[noybin]

Thank you.

Is there any way to do this for specific hosts? or It would filter events for every hosts?

[tskinnerivsec]

The only ways that I can think of to apply it to some hosts and not others would be to assign a custom sourcetype to the windows event logs coming from the servers that you want to filter out, and in your props/transforms, assign the transform to that custom sourcetype. Or, you could stand up a heavy forwarder between the UFs and the Indexers, deploy the transform to the heavy forwarder (assigned to the normal sourcetypes) and then, forward just the UFs that you want filtered to the Heavy Forwarders, which will parse the events, then forward on to the indexers. If you create a custom sourcetypes, I would call it something like WinEventLog://Security_filtered, so that you can still perform searches like sourcetype = WinEventLog://Security* and get all your events, whether they were filtered or not.

[noybin]

Thanks for your answer, but in the inputs.conf specification says that there can be filters by event id:
http://docs.splunk.com/Documentation/Splunk/6.2.5/admin/inputsconf

# EventLog filtering
# 
# Filtering at the input layer is desirable to reduce the total processing load
# in network transfer and computation on the Splunk nodes acquiring and
# processing the data.

whitelist =  | key=regex [key=regex]
blacklist =  | key=regex [key=regex]

* The base unumbered whitelist and blacklist support two formats, a list of
  integer event IDs, and a list of key=regex pairs.

# Formats:

* Event ID list format:
  * A comma-seperated list of terms.
  * Terms may be a single event ID (e.g. 6) or range of event IDs (e.g. 100-200)
  * Example: 4,5,7,100-200
    * This would apply to events with IDs 4, 5, 7, or any event ID between 100
      and 200, inclusive.

[tskinnerivsec]

I've always used a transform for this, but now it looks like UF v. 6.1 introduced this type of blacklisting as a new feature. Check out this blog post for what looks like a good example:

http://blogs.splunk.com/2014/05/23/controlling-4662-messages-in-the-windows-security-event-log/