I'm attempting to update our certs between our universal forwarders (UF) and indexers in our test environment. I believe I have the certs properly generated and in place. But when the UF attempts to forward, we see this error:
10-19-2018 08:13:14.661 -0600 WARN SSLCommon - Received fatal SSL3 alert. ssl_state='SSLv3 read server hello A', alert_description='handshake failure'.
10-19-2018 14:17:44.863 +0000 WARN SSLCommon - Received fatal SSL3 alert. ssl_state='SSLv3 read client hello C', alert_description='handshake failure'.
10-19-2018 14:17:44.863 +0000 ERROR TcpInputProc - Error encountered for connection from src=nn.nn.nn.nn:38438. error:1408A0C1:SSL routines:ssl3_get_client_hello:no shared cipher
This leads me to believe that the cipherSuite needs to be updated ...
indexer server.conf - ( Splunk 7.1.3 ]
[sslConfig]
sslVersions = tls1.2
sslVersionsForClient = tls1.2
cipherSuite = ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:ECDH-ECDSA-AES256-GCM-SHA384:ECDH-ECDSA-AES128-GCM-SHA256:ECDH-ECDSA-AES128-SHA256:AES256-GCM-SHA384:AES128-GCM-SHA256:AES128-SHA256
( etc/system/local/inputs.conf under [SSL] )
cipherSuite = ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:ECDH-ECDSA-AES256-GCM-SHA384:ECDH-ECDSA-AES128-GCM-SHA256:ECDH-ECDSA-AES128-SHA256:AES256-GCM-SHA384:AES128-GCM-SHA256:AES128-SHA256:TLSv1+HIGH:TLSv1.2+HIGH:@STRENGTH
UF - Splunk 6.6.4 - etc/system/default/server.conf
[sslConfig]
cipherSuite = ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256
etc/system/default/outputs.conf
[tcpout]
sslVersions = tls1.2
cipherSuite = ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:AES256-GCM-SHA384:AES128-GCM-SHA256:AES128-SHA256:ECDH-ECDSA-AES256-GCM-SHA384:ECDH-ECDSA-AES128-GCM-SHA256:ECDH-ECDSA-AES256-SHA384:ECDH-ECDSA-AES128-SHA256
I've been using this link to generate and set up the new forwarding certs.
https://wiki.splunk.com/images/f/fb/SplunkTrustApril-SSLipperySlopeRevisited.pdf
From the splunk docs i have observed server.conf ciphersuite is different from inputs.conf and outputs.conf. Check your cipheresuite.
https://docs.splunk.com/Documentation/Splunk/7.1.3/Security/Ciphersuites
Hi,
While looking at $SPLUNK_HOME/default/etc/system/default/inputs.conf
it has below ciphersuite, can you please remove cipherSuite
from [SSL]
stanza in $SPLUNK_HOME/default/etc/system/local/inputs.conf
on Indexer so that it will use default cipherSuite.
cipherSuite = ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256
I will 2nd this. Is there a reason you aren't using the default CipherSuite?
Thank you ...
The default didn't work, so I went back and added the content at the end of the list as I'd seen that had solved different SSL issues when I upgraded beyond 6.5 ( guessing on the version )
I've reverted everything back to the default and I'm still getting the same errors.
Couple of things to check, is the sslPassword
same on both UFs and Indexer?
And stanza name in outputs.conf is [tcpout]
instead of [tcpoutput]
Indexers should be configured to accept encrypted data, meaning, inputs.conf on indexers should have a stanza defined as [splunktcp-ssl:<port>]
* Set to the port on which the forwarder sends the encrypted data
Thank you for your comments ...
the stanza is definitely [tcpout] ... the error was due to my typing this out in haste. Indexers are definitely listening on the splunktcp-ssl port I configured. I'll edit the post to the correct setting.
[splunk@somewhere ~]$ lsof -Pi :9998
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
splunkd 14568 splunk 47u IPv4 150502287 0t0 TCP *:9998 (LISTEN)
[splunk@somewhere ~]$ /opt/splunk/bin/splunk btool inputs list splunktcp-ssl
[splunktcp-ssl://9998]
_rcvbuf = 1572864
evt_dc_name =
evt_dns_name =
[SSL]
password = +-------redacted encrypted password ----+
rootCA = $SPLUNK_HOME/etc/slave-apps/_cluster/auth/cacert.crt
serverCert = $SPLUNK_HOME/etc/slave-apps/_cluster/auth/secidx.pem