reposting for a user over on the forums:
I bounced my indexer and now my forwarders are unable to connect. I just upgraded this indexer to 4.1.2 (although it's been working for hours since the upgrade).
The error in the splunkd.log is: TcpInputProc - Stopping all listening ports. Queues blocked for more than 300 seconds
What does this mean? I can't seem to find any relevant info. This is a 64b OpenSuse10.3 box.
For whatever reason, indexing is not going on, so the indexer has stopped accepting data on input ports. It might be because you're out of space, or your indexes are otherwise locked out (possibly because permissions aren't enabled). It's seems likely there will be other errors in splunkd.log, or clues in metrics.log
I was actually working with this user in the official support channels, I think the problem was his inputs.conf had the queues specified with
[inputstanza]
queue = parsingQueue
Looking at his metric queue diagnostics he was getting over 1000 events in his parsing queue even at fairly slow times. We removed the queue setting entirely and he's no longer getting queue blocked message, queue's dont have to be explicitly specified in 4.x. as splunk should default to parsing queue for all stanzas.
For whatever reason, indexing is not going on, so the indexer has stopped accepting data on input ports. It might be because you're out of space, or your indexes are otherwise locked out (possibly because permissions aren't enabled). It's seems likely there will be other errors in splunkd.log, or clues in metrics.log