Splunkd will not start and splunkd.log shows this error message:
"ERROR SQLitePersistentStorageImpl - Error executing: create table keyvaluepairs_t(primarykey text constraint primary_c unique,secondary1 text, secondary2 text, secondary3 text, value text); Msg=database is locked"
environment:
Splunk 6.4.1
Linux
Splunk install sits on an NFS
Splunk indexed data sits on iscsi
In this case the nfslock daemon was not running on the nfs server (10.10.10.25).
on the indexer the /var/log/messages file shows errors like:
kernel: lockd: cannot monitor 10.10.10.25
To resolve this issue:
start the nfslock service on the NFS server.
Delete $SPLUNK_HOME/var/lib/splunk/persistentstorage/seqno_db on the Splunk instance.
Start Splunk
In this case the nfslock daemon was not running on the nfs server (10.10.10.25).
on the indexer the /var/log/messages file shows errors like:
kernel: lockd: cannot monitor 10.10.10.25
To resolve this issue:
start the nfslock service on the NFS server.
Delete $SPLUNK_HOME/var/lib/splunk/persistentstorage/seqno_db on the Splunk instance.
Start Splunk