Hello
I have a Universal Forwarder that acts as an intermediary forwarder between about 200 other UFs and the Indexer. Data from the client UFs gets delayed or not sent at all and digging through the logs, I noticed this:
On the client UF:
09-29-2012 05:42:07.620 -0400 WARN TcpOutputFd - Connect to 10.xx.xx.60:9997 failed. No connection could be made because the target machine actively refused it.
09-29-2012 05:42:07.620 -0400 ERROR TcpOutputFd - Connection to host=10.xx.xx.60:9997 failed
09-29-2012 05:42:07.620 -0400 WARN TcpOutputProc - Applying quarantine to idx=10.xx.xx.60:9997 numberOfFailures=52
On the intermediary UF:
splunkd.log:
09-29-2012 06:35:48.722 -0400 INFO TailingProcessor - Could not send data to output queue (parsingQueue), retrying...
09-29-2012 06:35:49.504 -0400 INFO TailingProcessor - ...continuing.
metrics.log:
09-29-2012 06:27:24.509 -0400 INFO Metrics - group=queue, name=fschangemanager_queue, max_size_kb=5120, current_size_kb=0, current_size=0, largest_size=0, smallest_size=0
09-29-2012 06:27:24.509 -0400 INFO Metrics - group=queue, name=indexqueue, max_size_kb=10240, current_size_kb=0, current_size=0, largest_size=0, smallest_size=0
09-29-2012 06:27:24.509 -0400 INFO Metrics - group=queue, name=nullqueue, max_size_kb=10240, current_size_kb=0, current_size=0, largest_size=0, smallest_size=0
09-29-2012 06:27:24.509 -0400 INFO Metrics - group=queue, name=parsingqueue, blocked=true, max_size_kb=512, current_size_kb=511, current_size=409, largest_size=445, smallest_size=163
09-29-2012 06:27:24.509 -0400 INFO Metrics - group=queue, name=splunktcpin, max_size_kb=500, current_size_kb=423, current_size=218, largest_size=442, smallest_size=152
09-29-2012 06:27:24.509 -0400 INFO Metrics - group=queue, name=tcpin_queue, max_size_kb=10240, current_size_kb=0, current_size=0, largest_size=0, smallest_size=0
09-29-2012 06:27:24.509 -0400 INFO Metrics - group=queue, name=winparsing, max_size_kb=10240, current_size_kb=0, current_size=0, largest_size=1, smallest_size=0
I already updated the etc\system\local\server.conf on the intermediary UF adding this:
[queue]
maxSize = 10MB
[queue=parsingqueue]
maxSize = 10MB
[queue=splunktcpin]
maxSize = 10MB
however, as you can see from the log, those two queues are the only ones that seem to not get affected, showing max_size_kb to be 500-512 KB, instead of the 10 MB I set. Any idea why? Or am I not even on the right track?
... View more