In the process of migrating to an indexes app instead of fixed /opt/splunk/etc/system/local/indexes.conf
, I did a search against the indexer.
As soon as I renamed indexes.conf prior to restart (and picking up indexes.conf elsewhere in directory structure -prior to download of app), a search against that indexer immediately reported - no data ( for any window of time).
As soon as I renamed it back - data became available again.
It appears that forwarders continued to send data to the indexer the whole time.
Is there shift in how and when Splunk reads these config files? Indexes.conf in particular.
That was not how I remember it working on older versions.
I'm not aware of any difference, but the best way to diagnose if you have different configuration when you change your setup is to use btool to give a before and after and if there is a difference you can run with debug so you can see where each configuration item is coming from. Splunk doesn't need to be running for this, so you can make changes and keep doing it until everything matches up.
./bin/splunk btool indexes list
./bin/splunk btool indexes list --debug
I like to run it into a vim buffer like so:
./bin/splunk btool indexes list --debug | vim -
The dash makes vim read stdin into a new buffer.
Another alternative is to diff it
./bin/splunk btool indexes list > /tmp/indexes.old
mv ... # make your changes
./bin/splunk btool indexes list > /tmp/indexes.new
diff -u /tmp/indexes.{old,new} | less