SplunkWeb Certificates Issue

[MHibbin]

Recently I noticed I couldn't gain access to Splunkweb on one of Splunk installations. The installation was running fine when I used it previously, and then next day I was met with a certificates issue.

When I stop and start the services I see the following output:

# ./splunk start

Splunk> Australian for grep.

Checking prerequisites...
        Checking http port [8000]: open
        Checking mgmt port [8089]: open
        Checking configuration...  Done.
        Checking index directory...
        Validated databases: _audit _blocksignature _internal _thefishbucket history main summary
        Done
Success
        Checking conf files for typos...
All preliminary checks passed.

Starting splunk server daemon (splunkd)...
                                                           [  OK  ]
Done.Starting splunkweb... Generating certs for splunkweb server
Generating a 1024 bit RSA private key
........++++++
......++++++
writing new private key to 'privKeySecure.pem'
-----
Signature ok
subject=/CN=dev/O=SplunkUser
Error opening CA Certificate ca.pem
22576:error:02001002:system library:fopen:No such file or directory:bss_file.c:356:fopen('ca.pem','r')
22576:error:20074002:BIO routines:FILE_CTRL:system lib:bss_file.c:358:
unable to load certificate
Command failed (ret=1), exiting.

I'm not sure what has gone wrong here... any advice would be appreciated.

Thanks in advance,

MHibbin

[araitz · Accepted solution]

Looks like you are trying to generate a certificate against a non-existant root CA. You might need to generate a new root CA. Try reading the following section of the docs:

 http://docs.splunk.com/Documentation/Splunk/4.3.1/admin/Secureaccesstoyoursplunkserverwithssl#Genera... 

[MHibbin]

I can't think of any changes I made to etc or etc/auth... I normally restrict my changes to etc/apps or etc/system. :S I must have done something outside of Splunk then (though I'm not sure what). Thanks anyway!

[araitz]

I don't think that there are many (if any) scenarios where Splunk will remove ca.pem. Given that Splunk was trying to generate a new cert on start up, it seems that the server.pem file went missing as well. Any recent changes to server.conf or $SPLUNK_HOME/etc, specifically $SPLUNK_HOME/etc/auth?

[MHibbin]

That corrected the issue... Do you know how I might find the cause of the issue. i.e. what to look for in logs (splunk or system)?