Hi there,
I would like to initially bulk fill my kvstore with around 3.000.000 entries.
AFAIK the REST API allows to add one entry at a time. Not taking the overhead into account, this takes way too long.
Therefore, I need a direct connection to the kvstore (mongo db 2.6.7-splunk). My first task then would be: "getting a list of all database objects (show dbs)".
I tried using mongo shell 3.0.3. The connection could be established (to the db admin). But I am not allowed to do:
show db
"errmsg" : "not authorized on admin to execute command { listDatabases: 1.0 }",
"code" : 13
But I am connected not using any pem/key file.
With node.js and the package mongodb, I use these options, since I found them in mongod.log:
options: { net: { port: 8191, ssl: { PEMKeyFile: "xxx\etc\auth\server.pem", PEMKeyPassword: "<password>", mode: "preferSSL" } }, replication: { oplogSizeMB: 1000 }, security: { keyFile: "xxx\var\lib\splunk/kvstore\mongo\splunk.key" }, setParameter: { enableLocalhostAuthBypass: "0" }, storage: { dbPath: "xxx\var\lib\splunk/kvstore\mongo", smallFiles: true }, systemLog: { timeStampFormat: "iso8601-utc" } }
But same with this approach:
AssertionError: null == {"name":"MongoError","message":"not authorized on admin to execute command { listDatabases: 1 }","ok":0,"errmsg":"not authorized
My questions so far:
Thanks - Björn
b) you can create a lookup for this collection, save your data into CSV file and migrate it from CSV into your kvstore collection by issuing a search like "| inputlookup myfile.csv | outputlookup mykvstorecollection_lookup"
2 & 3. That is not supported. Even if you will find out how to make 2 and 3 work - there are will be a huge chance that not everything will work as you expect or something can be broken with following updates. Please let us know if you have some other reasons (other than #1) why would you like to have #2 and #3.
b) you can create a lookup for this collection, save your data into CSV file and migrate it from CSV into your kvstore collection by issuing a search like "| inputlookup myfile.csv | outputlookup mykvstorecollection_lookup"
2 & 3. That is not supported. Even if you will find out how to make 2 and 3 work - there are will be a huge chance that not everything will work as you expect or something can be broken with following updates. Please let us know if you have some other reasons (other than #1) why would you like to have #2 and #3.
Thanks for you input on this. I actually missed the batch_save endpoint.
For #2 and #3: One could use the kv store as an interface to master data systems. Moreover in a scenario where (critical) process data is stored within the kv store the customer might be interested in having a non-proprietary access to this kind of data.