Is it performance intensive for a splunk light forwarder to look for a long list of log file names, some of which will never exist?

[tpsplunk]

we are trying to come up with a solution that allows us to push a fairly generic inputs.conf to each of our Light Weight Forwarders. The goal is to be able to push the same inputs.conf to any server and have the regex in the inputs.conf find and send the appropriate logs to the indexers. this allows us to make sure a new server is always pushing logs to the indexers without manual intervention. It has the added bonus that even if the server is repurposed from one application type to another and begins to write to a completely new log name the logs will still get sent to the indexers.

one solution for doing this is to create a list of all of our application log names (lets say there are about 100 of these) and put this in the inputs.conf that gets pushed out to each LWF. Each LWF would then check its log directory for each of the 100 log names, even though that server may never actually find more than about 5 of the 100 log names.

Is this a bad idea? is it performance intensive to continually look for file names that may never exist?

[gkanapathy · Accepted solution]

It is not particularly demanding to look for 100 files on a local disk. I would only worry about it if you were looking at a few hundred thousand files, and in that case you would probably worry more about latency (discovering the file if and when it does show up) than about load on the system.