I found an answer myself before posting the question. I posted it anyway; maybe somebody facing the same problem will find it useful.
The answer is based on Windows 10; much of this will be different in Linux, obviously.
Trouble is, the whole certificate validation is obscure by design, so you won't know what you've done wrong until you've fixed all issues and the validation succeeds. I hope the following solution covers it all, but I can't be sure. While struggling to get things done, I may have changed something not mentioned here, simply because it seemed irrelevant at the time.
1. Trust two certicates
In C:\Program Files\Splunk\etc\auth , there are two relevant certificates:
server.pem : this appears to be the certificate used by splunkd
ca.pem : this appears to be a root certificate that is necessary to get the certificate chain complete
Import both certificates into your computer's certificate store. Every certificate vendor will tell you how to. For Windows 10 instructions, just google 'mmc import certificate'.
There are a few pitfalls here:
By default, the file browser of the Certificate Import Wizard filters on a number of known file extensions: CER, CRT, PFX... but not PEM. This seems to suggest PEM is not an accepted format. This is not true; just change the file type filter to 'All files (*.*)' and select the desired file.
'Automatically select the certificate store based on the type of certificate' seems like a convenient option in the Certificate Import Wizard, but it's not. I really had to specify the right certificate store myself to get things working. In the end, I had both certificates present in two different stores, which is probably more than strictly necessary, but it doesn't seem to hurt. Like I said earlier, I have little experience with certificates.
Trusted Root Certification Authorities
Intermediate Certification Authorities
2. Use the right host name
Look at the details of Splunk's self-signed certificate. You can either do this with a web browser (navigate to https://localhost:8089 and drill through the security warnings until you get to see the certificate) or from command line:
"C:\Program Files\Splunk\bin\splunk.exe" cmd openssl s_client -connect localhost:8089
Notice the certificate is issued to the following common name (CN): SplunkServerDefaultCert.
This name must match the host name in the URL you are using to access Splunk. https://localhost:8089 will not do; it must be https://SplunkServerDefaultCert:8089
It is possible to let your machine treat SplunkServerDefaultCert as an alias of localhost. The easiest way to accomplish this is to edit your hosts file. In Windows 10, this file is typically located in this folder: C:\Windows\System32\drivers\etc
Open the file in a text editor (e.g. Notepad++) and add the following line:
127.0.0.1 SplunkServerDefaultCert
Note: the editor must be running as administrator, otherwise you will not be able to save your changes.
In a web browser, verify the URL works: https://SplunkServerDefaultCert:8089
... View more