Hi,
I must confess I'm still not understanding how wildcards work in inputs.conf. I've got a clustered application, with five instances on one server. The instances are named live-1,live-2,live-3,staging-1,staging-2. They're all located in /opt/INSTANCE_NAME
I'm trying to monitor all the live instances
[monitor:///opt/foo/live-*/logs/]
index = foo_live
sourcetype = log4j
crcSalt = <SOURCE>
blacklist= (\.(gz|bz2|z|zip)$)
The problem with this stanza is that everything under /opt/foo will be listed by 'splunk list monitor'. More than 16000 files... Including everything in /opt/foo/staging-[12] and /opt/foo/whatever. I don't understand how that's possible, since none of those paths include the "live-" part, but anyway...
Problem number 2 is that nothing will actually be forwarded to the indexer by this.
The documentation seems pretty straigh-forward on this, so I really don't understand why it isn't working. If I list every directory as individual stanzas, the forwarding will work as expected. However, I would really need a generic solution to match all future environments as well. (Several applications, several instances.)
Replying to myself, in case someone has an interest in the answer:
The problem with splunk list monitor
was encountered in a Universal Forwarder version 4.2.3.
This seems to have been fixed in 4.3.1 (or in between). I found this out by first comparing the output of a 4.2.3 and 4.3.1 Universal forwarder, then confirmed by upgrading the 4.2.3 SUF to 4.3.1.
In other word, splunk list monitor
will only list the logs matching the monitor
-stanza.
Replying to myself, in case someone has an interest in the answer:
The problem with splunk list monitor
was encountered in a Universal Forwarder version 4.2.3.
This seems to have been fixed in 4.3.1 (or in between). I found this out by first comparing the output of a 4.2.3 and 4.3.1 Universal forwarder, then confirmed by upgrading the 4.2.3 SUF to 4.3.1.
In other word, splunk list monitor
will only list the logs matching the monitor
-stanza.
Are all of the logs named the same for each instance in their own directory? Maybe you can try this if that is so:
[monitor:///opt/foo/*]
index = (?:live-[1-5]{1}/logs/foo_live.log)$
sourcetype = log4j
crcSalt =
I've never put a RegEx expression in the monitor stanza and not certain that would work.
rgcurry,
Yes, to a degree. The application is running under Tomcat, so there are files named catalina.out and files with the date pattern foo.yyyy-mm-dd.log.
Not sure what you're trying to do with the regex in index, though... (?)