[monitor://\\njros1bva0597\d$\LogFiles\W3SVC1\*.log]
disabled = false
host = NJROS1BVA0621
alwaysOpenFile = 1
sourcetype = Image Importer Logs
With the stanza above, splunk reports the number of files is 3. There are only 2 files in the directory!! Also we can get it to actual index any of the files in the directory. There are only 2 files in the directory and they both end in .log
Please help
From the manual: "Warning: In Windows, you cannot currently use a wildcard at the root level." Which is what you are doing. And to answer cmonig, I believe that the double-backslash \\ is only required in whitelists and blacklists, not the monitor stanza itself. (And you don't necessarily need a backslash in a whitelist, as it is a regular expression.)
The following should fix the problem:
[monitor://\\njros1bva0597\d$\LogFiles\W3SVC1]
disabled = false
host = NJROS1BVA0621
alwaysOpenFile = 1
sourcetype = Image Importer Logs
whitelist=.log$
I don't understand why you can't use whitelists. If you can't, then you are stuck with monitoring everything in the directory or finding another way around this AFAIK.
I would personally not use a sourcetype with spaces, but that's up to you. Also, the number of files that Splunk is monitoring may also include directories in the path, I think. To see what is actually being monitored, try this
$SPLUNK_HOME\bin\splunk list monitor
And Splunk will give you a list of the files it is monitoring - it may be a longer list than you expect!
Good move on checking the internal logs for the tailing status.
Updated: you said in the comments: "I want the ROUTEDB log files to get one source type and UPDATEDB to get another source type. I don't want or need the remaining 10 files." You can override the sourcetype setting as needed in props.conf
- more info here in the middle of the page at Specify sourcetype for source. Perhaps this would help you so that you can use a whitelist and not violate the rule about multiple stanzas with the same path.
If the ultimate problem is the $ in your path name, can you create a link and use it instead? For example, create a link to the d$ directory named d.dollar, and then make the monitor stanza look like this:
[monitor://\\njros1bva0597\d.dollar\LogFiles\W3SVC1]
From the manual: "Warning: In Windows, you cannot currently use a wildcard at the root level." Which is what you are doing. And to answer cmonig, I believe that the double-backslash \\ is only required in whitelists and blacklists, not the monitor stanza itself. (And you don't necessarily need a backslash in a whitelist, as it is a regular expression.)
The following should fix the problem:
[monitor://\\njros1bva0597\d$\LogFiles\W3SVC1]
disabled = false
host = NJROS1BVA0621
alwaysOpenFile = 1
sourcetype = Image Importer Logs
whitelist=.log$
I don't understand why you can't use whitelists. If you can't, then you are stuck with monitoring everything in the directory or finding another way around this AFAIK.
I would personally not use a sourcetype with spaces, but that's up to you. Also, the number of files that Splunk is monitoring may also include directories in the path, I think. To see what is actually being monitored, try this
$SPLUNK_HOME\bin\splunk list monitor
And Splunk will give you a list of the files it is monitoring - it may be a longer list than you expect!
Good move on checking the internal logs for the tailing status.
Updated: you said in the comments: "I want the ROUTEDB log files to get one source type and UPDATEDB to get another source type. I don't want or need the remaining 10 files." You can override the sourcetype setting as needed in props.conf
- more info here in the middle of the page at Specify sourcetype for source. Perhaps this would help you so that you can use a whitelist and not violate the rule about multiple stanzas with the same path.
If the ultimate problem is the $ in your path name, can you create a link and use it instead? For example, create a link to the d$ directory named d.dollar, and then make the monitor stanza look like this:
[monitor://\\njros1bva0597\d.dollar\LogFiles\W3SVC1]
Thanks. I spoke with them again and we are going with the props.conf file. They acknowledged that splunk struggles with the $ in the path when you have a wildcard as well.
You are correct - you can't have more than one monitor on the same path. But if there are only text files and binaries in the directory, along with the log files - why would you have more than one monitor stanza with the same path?
First, they told me to use whitelists. But then, based on my experience, you can't have more than 1 monitor with the same path. Then they told me to use wildcards. I have been trying to get this simple example going before I tackle the real problem.
I have confirmed that the $ is the issue. I just don't know how to fix it.
monitor://\njros1bva0597\dshare\LogFiles\W3SVC1*.log
or
monitor://\njros1bva0597\dshare\LogFiles\W3SVC1\UPDATEDB*.log
All work just great.
It's our standard of creating shares like D$ that seems to be the issue.
I don't think I can use whitelists because the Splunk SE guys told me that I couldn't.
In the end, I have 20 files in the directory. 5 that are named UPDATEDB-MM-DD-YY.log and 5 that are named ROUTEDB-MM-DD-YY.log. The remaining 10 are text files and binaries.
I want the ROUTEDB log files to get one source type and UPDATEDB to get another source type. I don't want or need the remaining 10 files.
Hi,
I'm not sure if this is really necessary, but don't you have to escape the backslashes in the monitor stanza?
[monitor://\\njros1bva0597\\d$\\LogFiles\\W3SVC1\\*.log]
Also, you could try to remove the dot after the wildcard, and see if that helps:
[monitor://\\njros1bva0597\\d$\\LogFiles\\W3SVC1\\*log]
Good luck!
:-)
Cheers,
Christoph
Have you read these two similar questions:
http://splunk-base.splunk.com/answers/2775/regexs-and-windows-paths-in-inputsconf-and-propsconf
&
http://splunk-base.splunk.com/answers/26094/whats-the-syntax-for-monitoring-a-local-windows-director...
It probably gets really tricky when you are trying to use the administrative pathing to a different box. Have you considered installing a universal forwarder directly on the box you are trying to monitor files from?
escaping the $ did not work...didn't even recognize the input
went back to my original config that was in the question. Here's some more info from the tailing status
parent \njros1bva0597\d$\LogFiles\W3SVC1*.log
type File did not match whitelist '^\\njros1bva0597\d$\LogFiles\W3SVC1\[^\]*.log$'.
try escaping the dollar sign: \$
tried this
monitor://\njros1bva0597\d$\LogFiles\W3SVC1\*log
I think the problem is our share d$...i think when everything get converted to regex the dollar sign messes things up but we don't know to fix
we cannot use whitelisting due to another splunk limitation
for some reason the splunkbase is removing the backslashes in the path!!! Bottom line we want all files ending in .log in the W3SVC1 directory
This is the stanza
[monitor://\njros1bva0597\d$\LogFiles\W3SVC1*.log]
disabled = false
host = NJROS1BVA0621
alwaysOpenFile = 1
sourcetype = Image Importer Logs