Getting Data In

Blacklisting clarification

Voltaire
Communicator

I am attempting to blacklist all files that end with these extensions in my inputs.conf file. The blacklist is not working correctly. These are image files located under a /images directory on the web server. These files are Not in the same directory as the log file. They are directory entries in the nginx-access log file contents.
For example

/wd/code/websites/wd-current/www/images/* 
/wd/code/websites/wd-current/www/js/resources/*
/wd/code/websites/sample.it/www/resources/*

I have attempted a few different methods. Any suggestions?

[default]
host = oh.br0ther.com

[monitor:///var/log/nginx-access.log]
blacklist= \.(jpg|png|gif|mov|js|swf|mp4|jar|signed|flv)$
Tags (1)
0 Karma

sdaniels
Splunk Employee
Splunk Employee

In your file monitoring stanza above you are referencing a file and not a directory. If I wanted to monitor the images directory and blacklist all of the image files I would so something like this:

[monitor:///wd/code/websites/wd-current/www/images/*]
index = myindex
sourcetype = mysourcetype
blacklist= \.(jpg|png|gif|mov|js|swf|mp4|jar|signed|flv)$

http://docs.splunk.com/Documentation/Splunk/5.0/Data/Monitorfilesanddirectories

0 Karma

Voltaire
Communicator

Nice ! thats where I found the reference. Thank you!

0 Karma

sdaniels
Splunk Employee
Splunk Employee

Some examples here. You can use elipsis wildcards or *.

http://docs.splunk.com/Documentation/Splunk/5.0/data/Specifyinputpathswithwildcards

0 Karma

Voltaire
Communicator

Excellent, just modified inputs.conf on my forwarder. Would this command work if I wanted to exclude all files under the www directory in any of the subfolders? or do I have to add a different syntax ?
for example
[monitor:///wd/code/websites/wd-current/www/.../*]

Thank you!

0 Karma

lguinn2
Legend

Your stanza will monitor only the following files:

1 - files named /var/log/nginx-access.log

2 - files underneath a directory named /var/log/nginx-access.log

Since this is not the stanza that is monitoring the directories that you name, putting the blacklist here will not help.

I don't see anything wrong with your blacklist, it just needs to be moved so that it will be part of the proper monitor stanza.

Voltaire
Communicator

Thank you Lisa!

0 Karma

Voltaire
Communicator

I am trying to block files from being read by splunk in those directories.
Thank you.

0 Karma

martin_mueller
SplunkTrust
SplunkTrust

Are you trying to block files from being read by Splunk or to block specific lines of a logfile from being indexed?

0 Karma
Get Updates on the Splunk Community!

Join Us for Splunk University and Get Your Bootcamp Game On!

If you know, you know! Splunk University is the vibe this summer so register today for bootcamps galore ...

.conf24 | Learning Tracks for Security, Observability, Platform, and Developers!

.conf24 is taking place at The Venetian in Las Vegas from June 11 - 14. Continue reading to learn about the ...

Announcing Scheduled Export GA for Dashboard Studio

We're excited to announce the general availability of Scheduled Export for Dashboard Studio. Starting in ...