ERROR X509 - X509 certificate

antuitadmin · ‎02-06-2018

02-07-2018 09:50:52.078 +0530 ERROR X509 - X509 certificate (OU=Cloud Team,emailAddress=cloud-eng@splunk.com,ST=CA,O=Splunk Cloud,L=San Francisco,CN=input-prd-p-XXXXX.cloud.splunk.com) common name (input-prd-p-XXXXX.cloud.splunk.com) did not match any allowed names (prd-p-XXXXX.cloud.splunk.com)
02-07-2018 09:50:52.078 +0530 ERROR TcpOutputFd - Connection to host=54.211.66.144:9997 failed
02-07-2018 09:50:52.923 +0530 ERROR X509 - X509 certificate (OU=Cloud Team,emailAddress=cloud-eng@splunk.com,ST=CA,O=Splunk Cloud,L=San Francisco,CN=input-prd-p-XXXX.cloud.splunk.com) common name (input-prd-p-XXXXXX.cloud.splunk.com) did not match any allowed names (prd-p-XXXXX.cloud.splunk.com)
02-07-2018 09:50:52.923 +0530 ERROR TcpOutputFd - Connection to host=54.211.66.144:9997 failed
02-07-2018 09:50:52.924 +0530 WARN TcpOutputProc - Applying quarantine to ip=54.211.66.144 port=9997 _numberOfFailures=2
02-07-2018 09:50:55.722 +0530 INFO DC:DeploymentClient - channel=tenantService/handshake Will retry sending handshake message to DS; err=not_connected
02-07-2018 09:51:07.722 +0530 INFO DC:DeploymentClient - channel=tenantService/handshake Will retry sending handshake message to DS; err=not_connected
02-07-2018 09:51:15.813 +0530 WARN HttpPubSubConnection - Unable to parse message from PubSubSvr:
02-07-2018 09:51:15.813 +0530 INFO HttpPubSubConnection - Could not obtain connection, will retry after=78.647 seconds.

antuitadmin · ‎02-08-2018

It is a forwarder and I am using splunk cloud.

micahkemp · ‎02-07-2018

It looks like your indexer is configured to check client certificates and their names, and the forwarder sending logs doesn't have a certificate which matches the indexer's allowed names. Check these configurations on your indexer's inputs.conf:

requireClientCert = <bool>
* Determines whether a client must present an SSL certificate to authenticate.
* Full path to the root CA (Certificate Authority) certificate store.
* The <path> must refer to a PEM format file containing one or more root CA
  certificates concatenated together.
* Certificates with the same Common Name as the CA's certificate will fail
  this check.
* Defaults to false for self-signed and third-party certificates. If using the 
default certificates, this attribute defaults to true and will override an existing 
false setting.

sslCommonNameToCheck = <commonName1>, <commonName2>, ...
* Check the common name of the client's certificate against this list of names.
* If there is no match, assume that the Splunk instance is not authenticated
  against this server.
* This setting is optional.
* Defaults to no common name checking.
* requireClientCert must be set to true for this setting to work.

The fix may be as simple as adding this forwarder's common name to the list in sslCommonNameToCheck.

antuitadmin · ‎02-08-2018

Please find output of command [Since I am using Windows system, grep is replaced by find]

c:\Program Files\SplunkUniversalForwarder\bin>splunk btool inputs list --debug | find "requireClientCert"
c:\Program Files\SplunkUniversalForwarder\etc\system\local\inputs.conf requireClientCert = False

and

c:\Program Files\SplunkUniversalForwarder\bin>splunk btool inputs list --debug | find "sslCommonNameToCheck"
c:\Program Files\SplunkUniversalForwarder\etc\system\local\inputs.conf sslCommonNameToCheck = prd-p-XXXXXX.cloud.splunk.com, input-prd-p-XXXXXX.cloud.splunk.com

Please suggest.

micahkemp · ‎02-08-2018

Did you run that on the forwarder or indexer? The indexer is where your inputs.conf will be configured to receive from your forwarder, so you need to check the configuration in the indexer’s inputs.conf.

antuitadmin · ‎02-07-2018

Thank you for reply.

I did change in input.conf [etc\system\local & etc\system\defaults]

[default]
host = MyHostName01

sslCommonNameToCheck = prd-p-XXXXXX.cloud.splunk.com, input-prd-p-XXXXXX.cloud.splunk.com

But issue is same, also metioned "requireClientCert = False". issue remains

02-08-2018 10:50:07.635 +0530 ERROR X509 - X509 certificate (OU=Cloud Team,emailAddress=cloud-eng@splunk.com,ST=CA,O=Splunk Cloud,L=San Francisco,CN=input-prd-p-XXXXXX.cloud.splunk.com) common name (input-prd-p-XXXXXX.cloud.splunk.com) did not match any allowed names (prd-p-XXXXXX.cloud.splunk.com)

02-08-2018 10:50:07.635 +0530 ERROR TcpOutputFd - Connection to host=54.211.66.144:9997 failed

This is a vanila installation and I followed exactly mentioned steps in --> http://docs.splunk.com/Documentation/SplunkCloud/6.6.3/User/ForwardDataToSplunkCloudFromWindows so requireClientCert should be default false. Dont know why I am facing this issue.

micahkemp · ‎02-08-2018

Don't make changes in any default/ directory. The proper location to make changes is in one of the local/ directories. See the documentation regarding this concept.

Based on the fact that you made the changes in default/, I'm not surprised you see requireClientCert=false. You need to look at your configurations as a whole, not just one config file to see where requireClientCert=true may be set. Try this:

splunk btool inputs list --debug | grep requireClientCert

That command will fetch the configurations as they will be used (considering precedence), and include the name and location of the file where the current setting exists.