Splunk Search

Wildcards and Regex(s) in Windows OS (UF) Monitor Paths (inputs.conf)?

kfeagans_splunk
Splunk Employee
Splunk Employee

Without any examples of Windows UF Monitor Paths (Universal Forwarder), it's pretty tough to figure out just what works and what doesn't. For example, the mixing of regex and wildcards, especially on Window is daunting. (for me at least).

IE: inputs.conf

[monitor://C:\OnState\Instances\k*01\ControlCP\]
whitelist = CPlog\d{0,3}\.txt$

Directory listing looks like:
C:\OnState\Instances\kfeagans01\ControlCP\CPlog01.txt
C:\OnState\Instances\kfeagans01\ControlCP\CPlog02.txt
C:\OnState\Instances\kfeagans01\ControlCP\CPlog03.txt
etc.

This I couldn't get to work, and it's generally because of mixing wildcards and regex between the monitor path and whitelist. Am I wrong here? There is an order to wildcards/regex(s) that must be followed that is outlined here (but without Windows examples, it's not useful): Wildcards and regular expression metacharacters

Thanks, Kelly

1 Solution

kfeagans_splunk
Splunk Employee
Splunk Employee

I found a solution, from a very awesome Splunker (Sanford, FTW!!). A better recipe is to use the whitelist for regex and leave wildcards out of the monitor path altogether. Remember that wildcards (*) are generally greedy, and using regex is much more surgical and generally much faster for processing.

Example - pick up CPlogs in c:\OnState\Instances\kfeagans01\ConsoleCP\CPlog01.txt

[monitor://C:\OnState\Instances\]
whitelist = k[A-z]+\d{1,2}\\ConsoleCP\\CPlog\d{0,3}\.txt
sourcetype=test
index=test

So, basically whitelist the subdirectory tree that includes your wildcards, in this case using regex instead. Works awesome. As a matter of fact, I was able to expand this to include other trees as well to pick up more data from the subdirs underneath each userid:

Example - pick up logfiles in the following directories:
C:\OnState\Instances\kfeagans01\ConsoleCP\CPlog01.txt (etc)
C:\OnState\Instances\kfeagans01\RemoteDB\DBlog01.txt (etc)
C:\OnState\Instances\kfeagans01\Accumulator\log01.txt (etc)
C:\OnState\Instances\kfeagans01\RTAClient\log01.txt (etc)

[monitor://C:\OnStat\Instances\]
whitelist = k[A-z]+\d{1,2}\(ConsoleCP|RemoteDB|Accumulator|RTAClient)\(CPlog|DBlog|log)\d{0,3}\.txt
sourcetype=test
index=test

You can even follow that up with a props/transforms to change the sourcetype from test to match the source; IE, create sourcetypes ConsoleCP, RemoteDB, Accumulator, RTAClient.

--

Kelly

View solution in original post

kfeagans_splunk
Splunk Employee
Splunk Employee

I found a solution, from a very awesome Splunker (Sanford, FTW!!). A better recipe is to use the whitelist for regex and leave wildcards out of the monitor path altogether. Remember that wildcards (*) are generally greedy, and using regex is much more surgical and generally much faster for processing.

Example - pick up CPlogs in c:\OnState\Instances\kfeagans01\ConsoleCP\CPlog01.txt

[monitor://C:\OnState\Instances\]
whitelist = k[A-z]+\d{1,2}\\ConsoleCP\\CPlog\d{0,3}\.txt
sourcetype=test
index=test

So, basically whitelist the subdirectory tree that includes your wildcards, in this case using regex instead. Works awesome. As a matter of fact, I was able to expand this to include other trees as well to pick up more data from the subdirs underneath each userid:

Example - pick up logfiles in the following directories:
C:\OnState\Instances\kfeagans01\ConsoleCP\CPlog01.txt (etc)
C:\OnState\Instances\kfeagans01\RemoteDB\DBlog01.txt (etc)
C:\OnState\Instances\kfeagans01\Accumulator\log01.txt (etc)
C:\OnState\Instances\kfeagans01\RTAClient\log01.txt (etc)

[monitor://C:\OnStat\Instances\]
whitelist = k[A-z]+\d{1,2}\(ConsoleCP|RemoteDB|Accumulator|RTAClient)\(CPlog|DBlog|log)\d{0,3}\.txt
sourcetype=test
index=test

You can even follow that up with a props/transforms to change the sourcetype from test to match the source; IE, create sourcetypes ConsoleCP, RemoteDB, Accumulator, RTAClient.

--

Kelly

ppablo
Retired

No problem Kelly 🙂 and awesome, you got it! More karma points for you!

0 Karma

kfeagans_splunk
Splunk Employee
Splunk Employee

Oh duh ... check mark next to the answer. 🙂 Accepted?

kf

kfeagans_splunk
Splunk Employee
Splunk Employee

Hi Patrick .. thanks! How to accept the answer? I don't readily see a button or pulldown or?

Thanks!

kf

0 Karma

ppablo
Retired

Hi @kfeagans_splunk

Glad you found a solution to your question! Please be sure to accept your answer to your post so other people with similar issues will refer to it for help 🙂 Thanks!

Patrick

Get Updates on the Splunk Community!

Extending Observability Content to Splunk Cloud

Watch Now!   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to leverage ...

More Control Over Your Monitoring Costs with Archived Metrics!

What if there was a way you could keep all the metrics data you need while saving on storage costs?This is now ...

New in Observability Cloud - Explicit Bucket Histograms

Splunk introduces native support for histograms as a metric data type within Observability Cloud with Explicit ...