We have setup a heavy forwarder (for VMware app as a dc node) but we are getting following errors in splunkd.log. Instance is starting correctly.
-0700 ERROR TcpOutputFd - Read error. Connection reset by peer
Any feedback? Thanks.
Any update on this?
I have a similar problem...
can you post you outputs.conf? its probably an issue there
I try to secure master <-> indexer communication with server certificates signed by our own company rootCA.
Reason is: Forwarding Master-Data to indexers so master does not index any data itself.
I created certificates for my servers according to the splunk documentation.
For the sake of this example I will call the server certificate: servercert.pem
And I will call the rootCA certificate: rootcacert.pem
Lets start at the formatting of the certificates:
the servercert.pem looks like
- servercert in pem format
- privatekey in rsa format (encrypted with secret-key)
- subCAcert in pem format (yes, we have a subCA)
- rootCAcert in pem format
the rootcacert.pem looks like
On the master the outputs.conf looks like this
[tcpout]
defaultGroup = Splunk_Indexers
[tcpout:Splunk_Indexer]
server = indexer1:9997,indexer2:9997
[tcpout-server://indexer1.ex.amp.le.de:9997]
sslRootCAPath = /opt/splunk/etc/auth/splunkforwarder/rootcacert.pem
sslCertPath = /opt/splunk/etc/auth/splunkforwarder/servercert.pem
sslPassword = <secret-key>
sslVerifyServerCert = true
sslCommonNameToCheck = indexer1.ex.amp.le.de
[tcpout-server://indexer2.ex.amp.le.de:9997]
sslRootCAPath = /opt/splunk/etc/auth/splunkforwarder/rootcacert.pem
sslCertPath = /opt/splunk/etc/auth/splunkforwarder/servercert.pem
sslPassword = <secret-key>
sslVerifyServerCert = true
sslCommonNameToCheck = indexer2.ex.amp.le.de
On the indexers the inputs.conf (distributed to them over the cluster-bundle) looks like this
[SSL]
rootCA = /opt/splunk/etc/auth/receiver/rootcacert.pem
serverCert = /opt/splunk/etc/auth/receiver/servercert.pem
password = <secret-key>
[splunktcp-ssl:9997]
compressed = true
Site note: I created a directory called splunkforwarder and receiver for reason of understandig which certificate resides where on the system.
If I open splunkd.log on the master I find the following error:
ERROR TcpOutputFd - Read error. Connection reset by peer
On the indexers splunkd.log looks like this:
ERROR TcpInputProc - Error encountered for connection from src=:38953. error:140760FC:SSL routines:SSL23_GET_CLIENT_HELLO:unknown protocol
Help would be hugely apprechiated, because I work on this problem since monday!
inputs.conf:
[default]
host = xyz [splunktcp-ssl://9997]
disabled=0
outputs.conf: [tcpout]
defaultGroup = indexers
indexAndForward = 1 [tcpout:indexers]
disabled = false
autoLB = true
server = indexer1:9997, indexer2:9997, indexer3:9997
[tcpout-server://indexer1:9997]
disabled = 0
[tcpout-server:/indexer2:9997]
disabled = 0
[tcpout-server://indexer3:9997]
disabled = 0
You're going to want to post inputs.conf and outputs.conf here... The error is most likely caused by a configuration error (usually an accidental loop).