Hi to all,
I have several Forwarders on Windows that monitor more than 20k items each (folder and logs inside them).
In total I'm monitoring more than 200k logs.
This is my inputs.conf deployed on those forwarders:
[monitor://c:\FirstFolder\Log\service\System\20*\PROXY\...]
disabled = false
index=my_index
sourcetype=proxy
alwaysOpenFile=1
[monitor://c:\\FirstFolder\Log\service\System\20*\00*\*.log]
disabled = false
index=my_index
sourcetype=process
alwaysOpenFile=1
[monitor://c:\\FirstFolder\Log\service\System\20*\00*.log]
disabled = false
index=my_index
sourcetype=device
alwaysOpenFile=1
[monitor://c:\\FirstFolder\Log\service\System\20*\ERROR_*.log]
disabled = false
index=main
sourcetype=error
alwaysOpenFile=1
When I restart the forwarders, they does not monitor anymore the files that the forwarders read before, even if the log files will be written after the restart.
Hi @robertosegantin ,
Did you try cheking splunk internal logs? Are der any ERRORS in log_level?
index=_internal
We had similar issues with a unix forwarder. I has to monitor over a million of files. It stopped sending logfiles and had high CPU consumption. I assume you are struggling with the same issue as we did.
The UF doesnt like haveing lots of open files like you configured with alwaysOpenfile=1
Even if you get rid of this option I doubt that the UF will work.
Try using the batch://
option. It reads and deletes the file. if you dont want to delete the file on your server I suggest, that you write a script in which the files are copied to a temporary directory.
In the end you will have configuration like this (inputs.conf):
[batch://c:\FirstFolder\Log\service\System\20*\PROXY\...]
disabled = false
index=my_index
sourcetype=proxy
move_policy = sinkhole