Getting Data In

How do I find and remove strings in logs from the Forwarder?

brandonmcgrath1
New Member

Hello,

I'm trying to send some antivirus logs from the forwarder into Splunk.

The logs I'm sending have a tendency to spam, for example:

13/09/2018 16:06:53 No usable rule found    Blocked 192.168.0.40:53354  192.168.0.30:53 UDP     C:\Windows\System32\dns.exe NT AUTHORITY\SYSTEM
13/09/2018 16:06:54 No usable rule found    Blocked 192.168.0.40:52091  192.168.0.30:53 UDP     C:\Windows\System32\dns.exe NT AUTHORITY\SYSTEM
13/09/2018 16:06:54 No usable rule found    Blocked 192.168.0.40:49467  192.168.0.30:53 UDP     C:\Windows\System32\dns.exe NT AUTHORITY\SYSTEM
13/09/2018 16:06:54 No usable rule found    Blocked 192.168.0.40:53354  192.168.0.30:53 UDP     C:\Windows\System32\dns.exe NT AUTHORITY\SYSTEM
13/09/2018 16:06:55 No usable rule found    Blocked 192.168.0.40:52091  192.168.0.30:53 UDP     C:\Windows\System32\dns.exe NT AUTHORITY\SYSTEM
13/09/2018 16:06:56 No usable rule found    Blocked 192.168.0.40:53354  192.168.0.30:53 UDP     C:\Windows\System32\dns.exe NT AUTHORITY\SYSTEM
13/09/2018 16:06:57 No usable rule found    Blocked 192.168.0.40:52091  192.168.0.30:53 UDP     C:\Windows\System32\dns.exe NT AUTHORITY\SYSTEM
13/09/2018 16:06:58 No usable rule found    Blocked 192.168.0.40:56694  192.168.0.30:53 UDP     C:\Windows\System32\dns.exe NT AUTHORITY\SYSTEM
13/09/2018 16:06:59 No usable rule found    Blocked 192.168.0.40:56694  192.168.0.30:53 UDP     C:\Windows\System32\dns.exe NT AUTHORITY\SYSTEM

I want to be able to filter out lines in the log that say "No usable rule found".

I've tried adding the following to props.conf which I've copied into [C:\Program Files\SplunkUniversalForwarder\etc\system\local] directory, here is the line I added to props.conf:

[source:\path\to\log\log.txt]
SEDCMD-strip-detail-msg=^.*(listening on the port|[Nn]o usable rule found)*$

I have also tried messing with transform.conf too, but to no avail.

Any ideas guys?

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Universal forwarders cannot filter events. You must use a heavy forwarder or put your props.conf settings on your indexers.

The SEDCMD setting is not correct. It must be in the format s/<search regex>/<replacement string>/. Replacing the entire event string will not remove the event. To remove unwanted events, send them to nullQueue like so:

props.conf

[source:\path\to\log\log.txt] 
TRANSFORMS-set = setnull,setparsing

transforms.conf

[setnull]
REGEX = (listening on the port|[Nn]o usable rule found)
DEST_KEY = queue
FORMAT = nullQueue

[setparsing]
REGEX = .
DEST_KEY = queue
FORMAT = indexQueue
---
If this reply helps you, Karma would be appreciated.
0 Karma
Get Updates on the Splunk Community!

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...