Getting Data In

How to use wildcards in [monitor://...] along with whitelist pattern?

neiljpeterson
Communicator

This should be an easy one...

This works great

[monitor:///opt/tcserver/server/appname/logs]
whitelist = \.log$|\.log4j\.log\.
sourcetype = log4j

This does not

[monitor:///opt/tcserver/server/*/logs]
whitelist = \.log$|\.log4j\.log\.
sourcetype = log4j

I need to be able to use the wild card because I have an arbitrary number of wars running under .../server/... and want to index the logs from all of them.

I suspect this is due to an interplay between the splunk converting the wildcard in the monitor line to a whitelist regex pattern and the whitelist line itself, but I can't quite figure it out.

0 Karma

lmyrefelt
Builder

For "wildcard" i believe the following works;

.../

example

/path/to/something/where/.../we/.../need/wildcard/
0 Karma

cramasta
Builder

Wildcards in monitors give me headaches. Always running into them blacklisting files that I set in a different monitor. Ive stopped using them completely to stay sane.

Why not let it just recurse through the all the directories under server. Whitelist would make it only pick up the patterns matched

[monitor:///opt/tcserver/server/]
index = imsv1
whitelist = (\.log$|\.log4j\.log\.)
sourcetype = log4j

Or make the whitelist more complete to make sure they are under logs/

[monitor:///opt/tcserver/server/]
index = imsv1
whitelist = (logs\/.*\.log$|logs\/.*\.log4j\.log\.)
sourcetype = log4j
0 Karma

cramasta
Builder

Thanks for the edit MuS.

So can you not use a whitelist setting with a wildcard like neiljpeterson tried to do?

neiljpeterson yes you can specify the path in the whitelist regex. my second example should work.

0 Karma

MuS
SplunkTrust
SplunkTrust

From the docs http://docs.splunk.com/Documentation/Splunk/6.1.1/Data/Specifyinputpathswithwildcards#Wildcards_and_... :

When you specify wildcards in a file input path, Splunk Enterprise creates an implicit whitelist for that stanza. The longest wildcard-free path becomes the monitor stanza, and Splunk Enterprise translates the wildcards into regular expressions, as listed in the table above.

Additionally, Splunk Enterprise anchors the converted expression to the right end of the file path, so that the entire path must be matched.

0 Karma

neiljpeterson
Communicator

because I only want the log files in the logs folder. Does the regex in whitelist apply to the path as well as the file name?

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...