I have a set of logs that no longer appear to be being indexed. I had originally configured the monitor as follows...
[monitor://D:\jboss-4.0.2\server\appname\log]
disabled = false
host = ServerName
index = default
sourcetype = log4j
whitelist = (boot|stderr|stdout|server|appname|appname-web-audit)\.log
This configuration seemed to work fine. I was getting data from all of the logs as expected.
The problem was that I have log4j, access_combined and a custom log type in this same folder. I've tried a couple of different solutions and neither seemed to work. I'm not sure if my syntax is right or where to get feedback from splunk as to if they are any good or not.
First attempt:
inputs.conf
[monitor://D:\jboss-4.0.2\server\appname\log\(boot|stderr|stdout|server|appname|appname-web-audit).log]
disabled = false
followTail = 0
host = ServerName
index = default
sourcetype = log4j
[monitor://D:\jboss-4.0.2\server\appname\log\appname-virtualhost_(\d\d\d\d-\d\d-\d\d).log]
disabled = false
followTail = 0
host = ServerName
index = default
sourcetype = access_combined
Second attempt:
inputs.conf
[monitor://D:\jboss-4.0.2\server\appname\log]
disabled = false
followTail = 0
host = ServerName
index = default
props.conf
[source::D:\\jboss-4.0.2\\server\\appname\\log\\(boot|stderr|stdout|server|appname|appname-web-audit).log]
sourcetype = log4j
[source::D:\\jboss-4.0.2\\server\\appname\\log\\appname-virtualhost_(\d\d\d\d-\d\d-\d\d).log]
sourcetype = access_combined
Neither of these approaches seems to work as I would expect it to. Am I not configuring this correctly? Is there a way to get feedback from splunk on problems with the configuration? If I switch back to the original configuration it seems to start indexing again.
This configuration is being used with splunk v4.1 as a full forwarder running on Windows.
In Splunk 4.1.x, both 'attempt' patterns are intended to work. There may be some outstanding issues with followTail, so you may want to evaluate what results you get without that.
You may want to try turning on some debug settings to get more insight, or work with Splunk Support.
In etc/log-local.cfg, you could turn these on.
You can also turn them on/off interactively from Manager.
This might be of use: http://www.splunk.com/wiki/Community:Troubleshooting_Monitor_Inputs#File_inputs
In Splunk 4.1.x, both 'attempt' patterns are intended to work. There may be some outstanding issues with followTail, so you may want to evaluate what results you get without that.
You may want to try turning on some debug settings to get more insight, or work with Splunk Support.
In etc/log-local.cfg, you could turn these on.
You can also turn them on/off interactively from Manager.
This might be of use: http://www.splunk.com/wiki/Community:Troubleshooting_Monitor_Inputs#File_inputs
inputs.conf:
[monitor://D:\jboss-4.0.2\server\appname\log]
host = ServerName
_whitelist = (boot|stderr|stdout|server|appname|appname-web-audit)\.log$
props.conf:
[source::(?i)D:\\jboss-4.0.2\\server\\appname\\log\\(boot|stderr|stdout|server|appname|appname-web-audit)\.log]
sourcetype = log4j
[source::(?i)D:\\jboss-4.0.2\\server\\appname\\log\\appname-virtualhost_(\d\d\d\d-\d\d-\d\d)\.log]
sourcetype = access_combined
Note that I corrected the name vitualhost
that you had to virtualhost
.