Knowledge Management

do we need to automate KV store look ups?/

ramarcsight
Explorer

I am currently using CSV but due to the frequent activity of CSV which is there in my Search head, there is a bundle replication problem to all indexers under it, so I am thinking to move to KV store lookup.

I got the basic idea is to create a collection in the collection.conf

so my question: I am creating a collection on SH.

if I create a collection in collections.conf do I need to restart the Splunk ES? I have one SH and few indexers under it in a cluster.

and create a collection is enough or do I need to automate it?

what is the difference between normal lookup and automate lookup?

Will kv store will help me solve the problem as I am writing and deleting from the csv more frequently and consurrently ??

Tags (2)
0 Karma

kashz
Explorer

If I understand your problem statement, its basically being able to decide whether to use CSV vs KV Stores. For more clarity, take a look at this Splunk Documentation: http://dev.splunk.com/view/SP-CAAAEY7

Index based approach: Questions that I faced during a similar dilemma was being able to retrieve latest scores from the index, one solution is to use |stats latest(field1) by field2,

But KV stores are the best approach in my personal opinion,
Normal Lookup: So you can create a lookup file (kv-store) and create a lookup definition in the settings>lookup>lookup definitions view. But that lookup is not being applied. You need to apply that lookup say, to a data model for example.

Automated lookup: Once you define a lookup, create a lookup definition and then define an automatic lookup in the settings>lookup>automatic lookups, it applied INSTANTLY to the data that splunk ingests. PS. While creating an automatic lookup, you actually specify to what it applies sourcetype or host or source allowing splunk to use that information and apply your lookup to the data when it is being ingested.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...