All Apps and Add-ons

where splunk kv store data has been stored in splunk ?

manikandankasi
Explorer

Hi,
I'm new to splunk, i have couple of questions related to kv store.
use case 1:
i have created splunk app using splunk python sdk app name called "test_kvstore" and created one kv store name "test_data_kvstore" and added data to it aswell (using python sdk).
next i have defined kvlookup in splunk search head (web ui & gave global permission) then i just try to see the kvstore data in splunk search head so put this command in search head
|inputlookup test_data_kvlookup
getting this error in search head.
Error in 'inputlookup' command: Lookup failed because collection 'test_data_kvstore' in app 'test_kvstore' does not exist, or user 'manikandankasi' does not have read access

i couldn't get why im getting this error since the app and kvstore already defined.

Questions:
is it kvstore data specific to the app or its stored globally?
if i want see the test_data_kvstore then by default where its stored? and location?

any help on this much appriciated !!!

Thanks in Advance,
Mani

Tags (1)
0 Karma

DavidHourani
Super Champion

Hi @manikandankasi,

If you're new to Splunk I would advise you start by reading here to get some info about how the kv-store works and how to manage it to avoid having issues as it scales:
https://docs.splunk.com/Documentation/Splunk/7.2.6/Admin/AboutKVstore

Now on to you question, the location of the kvstore is in $SPLUNK_HOME/var/lib/splunk/kvstore and can easily be backed up in case you're worried about losing your data :
https://docs.splunk.com/Documentation/Splunk/7.2.6/Admin/BackupKVstore

As for creating a collection, you can follow one of those links :
https://docs.splunk.com/Documentation/Splunk/7.2.6/Knowledge/ConfigureKVstorelookups
http://dev.splunk.com/view/SP-CAAAEZJ

So yeah your configuration for collection.conf and transforms.conf is correct, you can also add the _keyfield to your list of fields in case you wish to be able to modify some of the event later on via rest or SPL query.

PS: you can use this app for managing your kvstore, makes life way easier https://splunkbase.splunk.com/app/3536/

Cheers,
David

harsmarvania57
SplunkTrust
SplunkTrust

Hi,

Splunk stores KVStore data in $SPLUNK_HOME/var/lib/splunk/kvstore directory, KVStore uses mongodb so in given directory you will able to find mongo directory and many more files in that directory which belongs to mongodb.

Have you created Lookup Definitions for KVStore on search head ?

0 Karma

manikandankasi
Explorer

yes i have created lookup in search head. these are the definitions for lookup under app/test_kvstore/local directory
filename: transforms.conf

[test_data_kvlookup]
collection = test_data_kvstore
external_type = kvstore
fields_list = field1,field2,field3

filename: collection.conf

[test_data_kvlookup]

did i missed anything in the configuration ? please advise me

Thanks,
Mani

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...