All Apps and Add-ons

Splunk Add-on for Amazon Web Services 3.0: Why doesn't blacklist seem to be working for an S3 input?

muebel
SplunkTrust
SplunkTrust

I'm working with the Splunk Add-on for AWS 3.0, and am having an issue with the S3 input.

The S3 input has a blacklist config directive available. The bucket I'd like to input has binaries mixed in with the actual logs I am interested in, and so I configured the blacklist to exclude this type of file (along with .conf) by this regex:

(\.bin$|\.conf$)

However, the input is still indexing files with sources that end in .bin. Has anybody worked a similar issue? Is my understanding of the S3 input blacklist config incorrect?

The documentation for the input is here http://docs.splunk.com/Documentation/AddOns/latest/AWS/S3 , with the description for the blacklist config as:

A regular expression to indicate the S3 paths that the Splunk platform should exclude from scanning.

This seems fairly straightforward, and typical for a splunk blacklist, so it leaves me quite confused. Thanks for any help!

kchen_splunk
Splunk Employee
Splunk Employee

Please use the following regex for the blacklist. In short, the regex should be a exact match, not only containing.

.*(\.bin$|\.conf$)
0 Karma

muebel
SplunkTrust
SplunkTrust

that seems to make sense, but I ended up modifying the blacklist to be:
bin$
And it was effective at preventing the .bin inputs at least.

0 Karma

mreynov_splunk
Splunk Employee
Splunk Employee

if I am not mistaken, you may need to add astrerisk before, as the regex matches the entire path. So something like this:

 (*\.bin*|*\.conf*)
0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...