All Apps and Add-ons

Splunk Add-on for Amazon Web Services 3.0: Why doesn't blacklist seem to be working for an S3 input?

muebel
SplunkTrust
SplunkTrust

I'm working with the Splunk Add-on for AWS 3.0, and am having an issue with the S3 input.

The S3 input has a blacklist config directive available. The bucket I'd like to input has binaries mixed in with the actual logs I am interested in, and so I configured the blacklist to exclude this type of file (along with .conf) by this regex:

(\.bin$|\.conf$)

However, the input is still indexing files with sources that end in .bin. Has anybody worked a similar issue? Is my understanding of the S3 input blacklist config incorrect?

The documentation for the input is here http://docs.splunk.com/Documentation/AddOns/latest/AWS/S3 , with the description for the blacklist config as:

A regular expression to indicate the S3 paths that the Splunk platform should exclude from scanning.

This seems fairly straightforward, and typical for a splunk blacklist, so it leaves me quite confused. Thanks for any help!

kchen_splunk
Splunk Employee
Splunk Employee

Please use the following regex for the blacklist. In short, the regex should be a exact match, not only containing.

.*(\.bin$|\.conf$)
0 Karma

muebel
SplunkTrust
SplunkTrust

that seems to make sense, but I ended up modifying the blacklist to be:
bin$
And it was effective at preventing the .bin inputs at least.

0 Karma

mreynov_splunk
Splunk Employee
Splunk Employee

if I am not mistaken, you may need to add astrerisk before, as the regex matches the entire path. So something like this:

 (*\.bin*|*\.conf*)
0 Karma
Get Updates on the Splunk Community!

Extending Observability Content to Splunk Cloud

Watch Now!   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to leverage ...

More Control Over Your Monitoring Costs with Archived Metrics!

What if there was a way you could keep all the metrics data you need while saving on storage costs?This is now ...

New in Observability Cloud - Explicit Bucket Histograms

Splunk introduces native support for histograms as a metric data type within Observability Cloud with Explicit ...