I'm working with the Splunk Add-on for AWS 3.0, and am having an issue with the S3 input.
The S3 input has a blacklist config directive available. The bucket I'd like to input has binaries mixed in with the actual logs I am interested in, and so I configured the blacklist to exclude this type of file (along with .conf) by this regex:
(\.bin$|\.conf$)
However, the input is still indexing files with sources that end in .bin. Has anybody worked a similar issue? Is my understanding of the S3 input blacklist config incorrect?
The documentation for the input is here http://docs.splunk.com/Documentation/AddOns/latest/AWS/S3 , with the description for the blacklist config as:
A regular expression to indicate the S3 paths that the Splunk platform should exclude from scanning.
This seems fairly straightforward, and typical for a splunk blacklist, so it leaves me quite confused. Thanks for any help!
Please use the following regex for the blacklist. In short, the regex should be a exact match, not only containing.
.*(\.bin$|\.conf$)
that seems to make sense, but I ended up modifying the blacklist to be:
bin$
And it was effective at preventing the .bin inputs at least.
if I am not mistaken, you may need to add astrerisk before, as the regex matches the entire path. So something like this:
(*\.bin*|*\.conf*)