It seems we are having several issues with our Splunk servers/architecture and I wanted to know if anyone else has had issues. If so, were you able to get them fixed? To give you an idea of our layout, we have two indexers and two search heads (all four are big physical boxes, Windows OS). We have 10 heavy forwarders spread round the world (these are virtual boxes, Windows OS and 1 Linux). We also have several hundred universal forwarders (Windows OS) sending data to the heavy forwarders for filtering. We are now on Splunk 6.0.x, but that hasn’t helped and I’m wondered if it might have hurt things. The kinds of issues we are having are:
Indexers stop receiving data (sometimes both – very bad!)
Universal forwarders loses connection to heavy forwarders (restarting Splunk on the heavy forwarders fixes the issues, but we haven’t found why the connection is dropped)
Sometimes the universal forwarders stop sending data (again, restarting Splunk fixes the problem)
There are times when the Splunk service is “running”, but Splunk is not actually running.
I’d just like to know if anyone else is having stability issues besides us. I thought that Splunk was supposed to be one of those rock solid applications that just runs, but we haven’t seen that. Maybe if it was running on Linux, but we don’t have that option.
... View more