Splunk Search

Lookup table vs Add Data

mistydennis
Communicator

Basic question: when using a static csv as a data source, what are the pros and cons of creating a new lookup table vs. adding data through the Settings section (Settings -> Add Data)?

I don't understand the difference between the two, can someone help me out?

0 Karma
1 Solution

somesoni2
SplunkTrust
SplunkTrust

When you add the static CSV with Add Data, the CSV data will be stored in specified index and would be tied to the timestamp of when it was indexed OR whatever timestamp field you've chosed (or Splunk found it). So, in order to search that data, you'll need to specify appropriate time range. Since it's indexed data, it'll cost Splunk license when ingested and its retention period will be bound to the retention period of the index.

When you add the CSV as lookup, it's still is a static csv data in Splunk and doesn't get indexed (so not license usage OR retention), or is timerange bound. It's a literal csv file available on all Search Head servers (assuming you use SH clustering OR use deployment server to push that lookup, else it'll only be available on the node you uploaded it). Please note that all lookups are part of knowledge bundle which SH sends to it's search peers (indexers), so if your static file is very large, you may see replication failures and errors during search.

Also, the way to you access the data will be different too (regular search vs lookup specific commands).

View solution in original post

somesoni2
SplunkTrust
SplunkTrust

When you add the static CSV with Add Data, the CSV data will be stored in specified index and would be tied to the timestamp of when it was indexed OR whatever timestamp field you've chosed (or Splunk found it). So, in order to search that data, you'll need to specify appropriate time range. Since it's indexed data, it'll cost Splunk license when ingested and its retention period will be bound to the retention period of the index.

When you add the CSV as lookup, it's still is a static csv data in Splunk and doesn't get indexed (so not license usage OR retention), or is timerange bound. It's a literal csv file available on all Search Head servers (assuming you use SH clustering OR use deployment server to push that lookup, else it'll only be available on the node you uploaded it). Please note that all lookups are part of knowledge bundle which SH sends to it's search peers (indexers), so if your static file is very large, you may see replication failures and errors during search.

Also, the way to you access the data will be different too (regular search vs lookup specific commands).

mistydennis
Communicator

I can always depend on you, @somesoni2 ! Thanks 🙂

0 Karma
Get Updates on the Splunk Community!

Welcome to the Splunk Community!

(view in My Videos) We're so glad you're here! The Splunk Community is place to connect, learn, give back, and ...

Tech Talk | Elevating Digital Service Excellence: The Synergy of Splunk RUM & APM

Elevating Digital Service Excellence: The Synergy of Real User Monitoring and Application Performance ...

Adoption of RUM and APM at Splunk

    Unleash the power of Splunk Observability   Watch Now In this can't miss Tech Talk! The Splunk Growth ...