Greetings,
I have set up 17 micro AWS boxes, One running a splunk 6.2.0 indexer, 8 with databases (8 mongo and 4 mongo and neo4j), 8 with Node.JS, and set them up with splunk 6.2.0 heavy forwarders monitoring relevant files and forwarding to the splunk receiver/indexer. The problem is but only 10 of the forwarding instances ever show up in the indexer.
The receiver seems to only see the most-recent 10 of them in the data summary. 6 of the newest boxes seem to just not show up in the data summary on the main receiver/indexer.
These 6 boxes appear to be configured properly: They have monitors on the relevant files when I 'splunk list monitor'. They also show the receiver as an "active forward" when I 'splunk list forward-server'. As far as I can tell they are set up the same as the other 10 boxes that work.
So where is the block/issue? Is there some 10-forwarder limit I am hitting? Is there a concurrent search limit manifesting as a 10-forwarder limit? Do I need to do a split across load balancers if the receiver and indexer are on the same machine?
Thanks in advance for any assistance.
... View more