Getting Data In

inputs.conf not picking up monitored file

MasterOogway
Communicator

I have what appears to be a simple monitor to watch for a specific file name with a regex to define the date stamped file.
The file in question is named, /log/blahblah_0.0.0.653_9110.log

On my LWF I have the following simple inputs.conf definition:

[monitor:///log/blahblah_.*\.log]
disabled=false
sourcetype=search-log

From ../splunkd.log I get the following error.
INFO TailingProcessor - No configurations match, will ignore path='/log/blahblah_0.0.0.653_9110.log'

I have also defined a monitor based on a whitelist with the same result.

[monitor:///log]
disabled=false
whitelist = blahblah_.*.log$
sourcetype=search-log

My question is, "why does Splunk not want to index this file? I have confirmed the Regex is defined correctly, so I believe that is not it. Most likely it is something VERY simple where I can't see the forest through the trees.

Thoughts?
MasterOogway

Tags (1)
0 Karma

mfrost8
Builder

I'm not sure if I have what you're looking for, but here's some more info that might be helpful.

First, I don't believe that the crcSalt line is going to help here. My understanding is that that just helps Splunk understand when it might or might not be looking at the same file (or the events in a file) that it's indexed previously. I don't think that has anything to do with adding the file to the list of files to monitor.

So what did "splunk list monitor" report? Did it show that at least /log was being monitored but the blahblah*.log file was not being monitored?

One important thing to note is that the line

[monitor:///log/blahblah_.*\.log]

is probably not doing what you think it is. Remember that in this context, there are some regex shortcuts that are being taken. If your file really is named

blahblah_0.0.0.653_9110.log

then

[monitor:///log/blahblah_.*.log]

(i.e. no escaped '.' since '.' is the literal period character) should work. Remember that this is creating an implicit whitelist. I'm actually not sure what the '.' would do, but I'd guess that it might actually look for a backslash followed by a dot.

I would have thought your whitelist version might have worked assuming you were using a version of newer version of Splunk where "whitelist" replaced the older "_whitelist" keyword.

Lowell and gkanapathy were kind enough to write up some nice details to a question I had a while back about the use of wildcards and regexes in inputs.conf at

http://splunk-base.splunk.com/answers/2775/regexs-and-windows-paths-in-inputsconf-and-propsconf

in case that might also provide some further clues.

0 Karma

brantramey
Explorer

We are having the same issue here. Many files are not being indexed. We have tried unsuccessfully to add the "crcSalt = ". Does anyone have any other solutions to this?

0 Karma

fox
Path Finder

Is it possible to open up your filter or are there lots of files in this directory? I preferred your second approach, but with a more simple regex like .*log for example. You could also try the line in the input file: "crcSalt = <SOURCE>"

0 Karma

MasterOogway
Communicator

I tried it with and without the crcSalt= line without luck. Short of naming the file explicitly I have tried just about every combination. Still looking for the "aha" answer.

0 Karma
Get Updates on the Splunk Community!

ICYMI - Check out the latest releases of Splunk Edge Processor

Splunk is pleased to announce the latest enhancements to Splunk Edge Processor.  HEC Receiver authorization ...

Introducing the 2024 SplunkTrust!

Hello, Splunk Community! We are beyond thrilled to announce our newest group of SplunkTrust members!  The ...

Introducing the 2024 Splunk MVPs!

We are excited to announce the 2024 cohort of the Splunk MVP program. Splunk MVPs are passionate members of ...