Getting Data In

Blacklist directory names not working

jackal242
Engager

I've added the following blacklist line:

[monitor:///usr/local/alert/logs]
blacklist = (bak|sqlsync|syncdb_log|sql_bak|WebCorder)

And yet I see in my sources the following files are being indexed for that monitor:

/usr/local/alert/logs/bak/ipdlog_new.12133 | 17,016
/usr/local/alert/logs/bak/ipdlog_new.18100 | 15,770
/usr/local/alert/logs/bak/ipdlog_new.9881 | 15,727 
/usr/local/alert/logs/WebCorder/FF27774.65391.72094_001_act_src.html | 1
/usr/local/alert/logs/WebCorder/FF27774.65391.72094_002_act_src.html | 1

So clearly the blacklist is not working.

Any help?

0 Karma
1 Solution

jackal242
Engager

I found the problem.

The problem is that I had already indexed the directory once without the blacklist and nothing removed those sources.

I thought removing the monitor and readding it with the blacklist would remove all the unwanted directories.

I had to do a "splunk clean eventdata" to get it to remove all the unwanted source directories that are now in the blacklist.

View solution in original post

dwaddle
SplunkTrust
SplunkTrust

(Some additional information for you)

The "Search" app's initial Dashboard (the Summary page) is generated by running several searches against the already-indexed data. The sources list runs the following search:

| metadata sources

This includes an up to date event count which is updated roughly every 25 seconds. More info on the metadata command is available at http://www.splunk.com/base/Documentation/latest/SearchReference/Metadata.

As you commented, removing an input from Splunk does not delete already indexed data from the index - it stops Splunk from indexing any additional data from those files.

dwaddle
SplunkTrust
SplunkTrust

Thanks Amrit for the additional detail here.

0 Karma

jackal242
Engager

I found the problem.

The problem is that I had already indexed the directory once without the blacklist and nothing removed those sources.

I thought removing the monitor and readding it with the blacklist would remove all the unwanted directories.

I had to do a "splunk clean eventdata" to get it to remove all the unwanted source directories that are now in the blacklist.

Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...