Getting Data In

Blacklist log events (not log filenames) using a string to limit events forwarding to Splunk Indexers

ejwade
Contributor

I have a dns log that is very chatty with internal requests (e.g. localserver5.internal). I would like to forward dns logs for external requests (maliciouswebsite.g.mail.com) but filter out (blacklist) local requests. I have found ways to blacklist log files, but not specific log entries.

Another note - the field extraction for this value occurs at the search head, not the app with the inputs file. For the host with the dns log, I created a custom app in our deployment server with a custom inputs.conf for monitoring this log. Not sure if that affects what's possible.

0 Karma

damien_chillet
Builder

Hi ejwade,

You can send specific events to the nullQueue to discard them at the indexer/heavy forwarder level.

In your case it would look like:

props.conf

[source::<bro_logs_source>]
TRANSFORMS-null= set null

transfroms.conf

[setnull]
REGEX = <your_regex> (for you something that deals with internal A record)
DEST_KEY = queue
FORMAT = nullQueue

You can have a read through the "Filter event data and send to queues" section at http://docs.splunk.com/Documentation/SplunkCloud/6.6.3/Forwarding/Routeandfilterdatad.

Hope that's helpful!

0 Karma

sshelly_splunk
Splunk Employee
Splunk Employee

Can upload some anonymized data ? This would occur at the indexer or heavy forwarder via regex most likely. A sample log (with more than a few whitelist and blacklist) events should suffice.

0 Karma

ejwade
Contributor

Sure. These are bro DNS logs, so they are tab delimited (I'll do comma below).

1511991992.963051,CE0oKO1yiHQLlxOB5g,10.10.10.10,47041,10.20.20.20,53,udp,13336,internal-srv.ewade.internal,C_INTERNET,1,A,0,NOERROR,T,F
1511991994.963051,CE0oKO1yweQLlxOB5g,10.10.10.10,47041,10.20.20.20,53,udp,13336,maliciouswebsite.g.mail.com,C_INTERNET,1,A,0,NOERROR,T,F

internal-srv.ewade.internal is the "A" record that we want to filter out, while maliciouswebsite.g.mail.com is the one we want to pass to Splunk. A RegEx would suffice, but I'm not sure where to do this or the syntax. "blacklist" under inputs.conf seems to only refer to filenames.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...