I work with UniversalForwarders (136 servers) sending data to a Heavy Forwarder Cluster (3 servers) that forward data to a Splunk Indexer (1 server).
On the 3 servers layers it's set useAck=true on configs.
I have just one windows server where I'm facing the following error.
12-30-2013 19:58:30.063 +0000 INFO TcpOutputProc - Connection to x.x.x.x:9997 closed. Read error. An existing connection was forcibly closed by the remote host.
12-30-2013 19:58:30.063 +0000 WARN TcpOutputProc - Possible duplication of events with channel=source::WinEventLog:Application|host::i-83f142a3|WinEventLog:Application|0, streamId=704141485450455387, offset=111562 on host=x.x.x.x:9997
12-30-2013 19:58:30.063 +0000 WARN TcpOutputProc - Possible duplication of events with channel=source::C:\Program Files\SplunkUniversalForwarder\var\log\splunk\splunkd.log|host::i-83f142a3|splunkd|30, streamId=0, offset=0 on host=x.x.x.x:9997
The others 135 servers are ok, just one is giving this error.
Any suggestion?
Yes linu1988, few seconds after restart my universal forwarder start to give me the same error message.
Hello,
It sounds like it might be a network issue. Here is a good document that describes the use of useACK, and describes a number of causes for the error you're getting.
http://docs.splunk.com/Documentation/Splunk/6.0.1/Forwarding/Protectagainstlossofin-flightdata
Did you try to restart the forwarder?
Now I have 3 servers facing same problem