Splunk v6.3 running on Windows Server 2008 R2
Server is a SearchHead
I am getting the socket errors pretty much non stop after SSL/TLS'ing everything.
https://localhost:8000 comes up fine and I can log in.
https://localhost:8089 comes up fine
Here are my configs:
============================================================================
ssl area from SERVER.CONF:
[sslConfig]
caCertFile = chain.pem
caPath = $SPLUNK_HOME/etc/auth/DOD
cipherSuite = HIGH
sslKeysfile = private_password.pem
sslKeysfilePassword = **HASHEDPASSWORD**
sslVersions = tls1.1, tls1.2
============================================================================
OUTPUTS.CONF:
[tcpout]
defaultGroup = myindexers
[tcpout:myindexers]
compressed = true
server = server1:9997,server2:9997
sslCertPath = $SPLUNK_HOME/etc/auth/DOD/private_password.pem
sslPassword = **HASHEDPASSWORD**
sslRootCAPath = $SPLUNK_HOME/etc/auth/DOD/chain.pem
sslVerifyServerCert = false
useACK = true
============================================================================
WEB.CONF:
[settings]
enableSplunkWebSSL = true
privKeyPath = etc/auth/splunkweb/web_private.pem
caCertPath = etc/auth/splunkweb/web_chain.pem
sslVersions = tls1.1, tls1.2
============================================================================
EXTRACTED FROM SPLUNKD.LOG
10-26-2015 11:56:49.705 -0400 INFO TcpOutputProc - Connected to idx=server2:9997 using ACK.
10-26-2015 12:03:09.391 -0400 WARN HttpListener - Socket error from 127.0.0.1 while idling: error:140760FC:SSL routines:SSL23_GET_CLIENT_HELLO:unknown protocol
10-26-2015 12:04:08.737 -0400 WARN HttpListener - Socket error from 127.0.0.1 while idling: error:140760FC:SSL routines:SSL23_GET_CLIENT_HELLO:unknown protocol
10-26-2015 12:04:18.947 -0400 INFO TcpOutputProc - Connected to idx=server1:9997 using ACK.
10-26-2015 12:04:20.213 -0400 WARN HttpListener - Socket error from 127.0.0.1 while idling: error:140760FC:SSL routines:SSL23_GET_CLIENT_HELLO:unknown protocol
10-26-2015 12:04:32.094 -0400 WARN HttpListener - Socket error from 127.0.0.1 while idling: error:140760FC:SSL routines:SSL23_GET_CLIENT_HELLO:unknown protocol
10-26-2015 12:04:44.570 -0400 WARN HttpListener - Socket error from 127.0.0.1 while idling: error:140760FC:SSL routines:SSL23_GET_CLIENT_HELLO:unknown protocol
============================================================================
when I double click on my public certificate, I see:
issued to: myservers FQDN
issued by: subordinate
expires in about two years
Certificate path:
root -
intermediate -
subordinate -
My private_password.pem file is my private key exported in openssl format with a password, begins with:
-----BEGIN RSA PRIVATE KEY-----
Proc-Type: 4,ENCRYPTED
My chain.pem is built as
myserver-
subordinate-
intermediate-
My web_private.pem is my private key exported in openssl format without a password
Hey rgsurfs!
I was receiving the same issue on a new Splunk instance with almost identical conf files as you. I also received the same socket error when my forwarders specified the cipherSuite server.conf. This went away when I commented out the cipherSuite line in server.conf on the forwarder side. Commenting out the cipherSuite line on the server side also resolves the socket errors in splunkd.log that you are seeing.
I see "dod" in the path to your certs, so I'm assuming that you have done FIPS_MODE=1 prior to starting Splunk. I guess this is a bug and you cannot use "cipherSuite" to force a cipher while also being FIPS compliant. FIPS will remove instances of SSL2/SSL3 regardless of what the CONFs say, so it's my assumption that the cipherSuite key is redundant and is causing issues.
Check out this post:
https://answers.splunk.com/answers/214531/trying-to-secure-splunk-browser-is-throwing-sslhan.html
SPL-92435 - Forcing TLS1.2 or TLS1.1 in server.conf with SPLUNK_FIPS does not work.
I checked the $SPLUNK_HOME/etc/splunk-launch.conf and there is no entry for FIPS_MODE=1.
I changed my server.conf file to:
[sslConfig]
sslKeysfilePassword = HASHEDPASSWORD
errors went away and now server is using the default splunk cert, vice my DOD cert.
Might you have any experience/guidance on using the DOD keys for this stanza:
sslKeysfile = private_password.pem - - is this just openssl formatted PEM key with password?
I cannot get splunk to read my private key. I keep getting errors: pem_read_bio:no start line
The sslKeysFile is your certificate chain formatted in: server.pem > serverPrivateKey.key > yourCAPublicKey.pem where "server.pem" is your certificate signed by your CA.
Make sure you create these using a command prompt or terminal (removing the possibility of any Unicode characters in the file).
For Windows:
type server.pem serverPrivateKey.key yourCAPublicKey.pem > myCertificateChain.pem
For Linux:
cat server.pem serverPrivateKey.key yourCAPublicKey.pem > myCertificateChain.pem
I specified my certificate chain in server.conf and left the sslKeysFilePassword setting alone. I did find I had issues with the certificate chain being in a folder deeper than /etc/auth (slow web server startup and sometimes it would just stop), so I left it in that folder.
[sslConfig]
allowSslCompression = false
caCertFile = myCertificateChain.pem
caPath = $SPLUNK_HOME/etc/auth
sslKeysfilePassword = HASHEDPASSWORD
sslVersions = *,-ssl2,-ssl3
Your /etc/system/local/web.conf should be:
[settings]
enableSplunkWebSSL = 1
sslVersions = *,-ssl2,-ssl3
httpport = 443
privKeyPath = etc/auth/mycerts/serverPrivateKey.key
caCertPath = etc/auth/mycerts/server.pem
I followed these directions for mine:
http://docs.splunk.com/Documentation/Splunk/6.2.0/Security/HowtoprepareyoursignedcertificatesforSplu...
So, my cacertfile and sslkeysfile are the same file ? format?