I am having problems with splunk configuration file precedence.
I have two inputs.conf in my splunk app. One in default and one in local. My reasoning is the sourcetype should be the same for all servers running this app. What I want to tune per server is the index that receives the logs. Some of these servers are in production which require logs for one year and some are in development which we only need logs for 30 days.
I put the sourcetype in the default/inputs.conf this should be the file no one needs to edit when using this app. I put the index value in the loca/inputs.conf, and here the system admin will specify if the server is dev or prod which will be input in the index= line
So far my logs are going into the main index which is not where I want them. I read the splunk wiki entry on precedence and unless I am reading it wrong I expect different results. I would expect splunk to combine the configuration stanzas for the log file and since I don't have conflicting configuration key values it should look something like this
[monitor:///some/path/file.log]
sourcetype=cool_log_type
index=dev
[monitor:///some/path/file.log]
sourcetype=cool_log_type
[monitor:///some/path/file.log]
index=dev
That looks like it should work, if the monitor statement is identical.
To see what Splunk thinks is being used, run this:
./splunk cmd btool inputs list --debug
That command will list out what Splunk is seeing as a merged inputs.conf, with the source on the left.
That looks like it should work, if the monitor statement is identical.
To see what Splunk thinks is being used, run this:
./splunk cmd btool inputs list --debug
That command will list out what Splunk is seeing as a merged inputs.conf, with the source on the left.
Found the error was in default/inputs.conf
one stanza was [monitor:///some/dir] the other was [monitor:/some/dir]. Splunk seemed to view those as separate entries and that caused the problem.