All Apps and Add-ons

Splunk Add-on for Amazon Web Services 3.0: Why doesn't blacklist seem to be working for an S3 input?

muebel
SplunkTrust
SplunkTrust

I'm working with the Splunk Add-on for AWS 3.0, and am having an issue with the S3 input.

The S3 input has a blacklist config directive available. The bucket I'd like to input has binaries mixed in with the actual logs I am interested in, and so I configured the blacklist to exclude this type of file (along with .conf) by this regex:

(\.bin$|\.conf$)

However, the input is still indexing files with sources that end in .bin. Has anybody worked a similar issue? Is my understanding of the S3 input blacklist config incorrect?

The documentation for the input is here http://docs.splunk.com/Documentation/AddOns/latest/AWS/S3 , with the description for the blacklist config as:

A regular expression to indicate the S3 paths that the Splunk platform should exclude from scanning.

This seems fairly straightforward, and typical for a splunk blacklist, so it leaves me quite confused. Thanks for any help!

kchen_splunk
Splunk Employee
Splunk Employee

Please use the following regex for the blacklist. In short, the regex should be a exact match, not only containing.

.*(\.bin$|\.conf$)
0 Karma

muebel
SplunkTrust
SplunkTrust

that seems to make sense, but I ended up modifying the blacklist to be:
bin$
And it was effective at preventing the .bin inputs at least.

0 Karma

mreynov_splunk
Splunk Employee
Splunk Employee

if I am not mistaken, you may need to add astrerisk before, as the regex matches the entire path. So something like this:

 (*\.bin*|*\.conf*)
0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

Splunk is officially part of Cisco

Revolutionizing how our customers build resilience across their entire digital footprint.   Splunk ...

Splunk APM & RUM | Planned Maintenance March 26 - March 28, 2024

There will be planned maintenance for Splunk APM and RUM between March 26, 2024 and March 28, 2024 as ...