Unable to use regex to index logs

[pimco_rgoyal]

Hi, I wish to configure splunk forwarder to pick logs from a directory that match any of the below patterns. Essentially anything that matches the regex "/^(jacket.)?[^\.]*-[^\.]*(.jvm)?.log$/". I tried to make below changes to inputs.conf but it is not working as expected. Can someone help guide how to debug further ?

• may start with “jacket.”
• must have at least one hyphen
• must end in “.log” or “.jvm.log”
• must not have any other “.” characters

Inputs.conf

[monitor:///base/apps/logs]
disabled = false
index = test
sourcetype = _json
whitelist = ^(jacket.)?[^\.]*-[^\.]*(.jvm)?.log$
blacklist = \.gz$

[FrankVl]

I took the liberty of editing your question, to put your code as code, to prevent special characters from dissapearing.

From inputs.conf spec:
whitelist =
* If set, files from this input are monitored only if their path matches the specified regex.

So the regex should match the path. Putting a ^ in it, which matches start of the string, and then only trying to match the filename is guaranteed to fail.

So for starters get rid of the ^ and put the folder instead:
/base/apps/logs/(jacket.)?[^\.]*-[^\.]*(.jvm)?.log$

Then, if there should only be 1 - in the name, the two classes left and right from it, should exclude the -. Also: the . characters need escaping. So we end up with:
/base/apps/logs/(jacket\.)?[^\.- ]*-[^\.-]*(\.jvm)?\.log$

Or, if you don't want to include the path, then specify that there shouldn't be any forward slashes in the match:
/(jacket\.)?[^\.- /]*-[^\.-/]*(\.jvm)?\.log$

[inventsekar]

may start with “jacket.”  -------------  ^(jacket.)?
must have at least one hyphen ---- (-)one hyphen added
must end in “.log” or “.jvm.log” ----- (.jvm)?.log$
must not have any other “.” characters -- [^\.]*

Please check -

  ^(jacket.)?[^\.]*-(.jvm)?.log$

[pimco_rgoyal]

Does not help resolve. Suspect this is some other issue and not something related to regex.

[inventsekar]

please check if its working..
^(jacket.)?log$

[DalJeanis]

First, you need to escape the periods, because in a regex, . means (roughly) "anything".

Second, you need to have a repeater on the character classes on either side of the hyphen, to allow more than one non-period character.

Third, we're adding hyphen to the list of things the first character class can't be, so that the system won't backtrack past the first hyphen. For the second one, we're NOT adding it, because we don't care if it's a hyphen or not.

This should be pretty efficient overall, finding matching patterns with only a single fail at each spot, and failing non-matching patterns completely at the first period with at most three retest steps.

Try...

 ^(jacket.)?[^-\.]*-[^\.]*(.jvm)?.log$

[pimco_rgoyal]

Does not help. Still not picking up logs as expected. What is strange is that my configuration is working fine as expected in Dev instance of our Splunk. I checked the props.conf to see if any diff that could result in this but found nothing.