All Apps and Add-ons

Splunk 6.6 upgrade seems to have permissions issues

earlhelms
Path Finder

After doing an rpm upgrade to 6.6 I'm having some pretty big issues that appear to be permission related.
Examples:
Failed to start KV Store process. See mongod.log and splunkd.log for details.

When I tried to search the splunkd log I received more errors...
•Error 'Could not find all of the specified destination fields in the lookup table.' for conf 'pan:config' and lookup table 'pan_vendor_info_lookup'.
•Error 'Could not find all of the specified destination fields in the lookup table.' for conf 'pan:hipmatch' and lookup table 'pan_vendor_info_lookup'.
•Error 'Could not find all of the specified destination fields in the lookup table.' for conf 'pan:system' and lookup table 'pan_vendor_info_lookup'.
•Error 'Could not find all of the specified destination fields in the lookup table.' for conf 'pan:threat' and lookup table 'pan_vendor_info_lookup'.
•Error 'Could not find all of the specified destination fields in the lookup table.' for conf 'pan:traffic' and lookup table 'pan_vendor_info_lookup'.
•Error 'Could not find all of the specified lookup fields in the lookup table.' for conf 'pan:threat' and lookup table 'threat_lookup'.

Fortunately, this is on my dev box.

0 Karma
1 Solution

earlhelms
Path Finder

I resolved my own issue...
/opt/splunk/bin/splunk createssl server-cert -d /opt/splunk/etc/auth -n server -c zzzz-zzzz.zzz.zzzz.com -l 2048
Note: I edited the FQDN in the above example

View solution in original post

earlhelms
Path Finder

I resolved my own issue...
/opt/splunk/bin/splunk createssl server-cert -d /opt/splunk/etc/auth -n server -c zzzz-zzzz.zzz.zzzz.com -l 2048
Note: I edited the FQDN in the above example

damode
Motivator

This will just recreate the default cert. In my case, we are using our own cert. Still getting the same error.
I checked file permission issue on Splunk.key file and that also seems fine.

0 Karma

earlhelms
Path Finder

A related link: https://answers.splunk.com/answers/457893/after-upgrading-to-650-kv-store-will-not-start.html

Which led me to this...
[root@zzz]# tail /opt/splunk/var/log/splunk/mongod.log
2017-05-08T14:35:11.400Z W CONTROL No SSL certificate validation can be performed since no CA file has been provided; please specify an sslCAFile parameter
2017-05-08T14:35:11.416Z F NETWORK The provided SSL certificate is expired or not yet valid.
2017-05-08T14:35:11.416Z I - Fatal Assertion 28652
2017-05-08T14:35:11.416Z I -
***aborting after fassert() failure

and this...
[root@zzzzz]# openssl x509 -enddate -noout -in ./server.pem
notAfter=Apr 13 20:57:57 2017 GMT

Still not entirely sure what to do

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...