Splunk Enterprise

How to index a csv/lookup file from SH to the indexer in a SH cluster

koshyk
Super Champion

hi
We have SH clustered environment and we have a lookup which is collected daily and this comes to SH and is present in all the SH cluster members correctly. But we have a requirement to index this (for legal and audit purposes), so the customer can see how the data looked like previously (say 20 days ago). What's the best way to index this csv/lookup file which is present in the SH cluster? when i thinking more and more it becomes complex and complex
(Options I thought so far: if I write an inputs.conf and pull this from a SH, this will have copy in every SH member and the inputs will be replicated too thus causing multiple copies to be indexed. Another alternative I thought is to run a Universal forwarder separately ..hmm..)

Any other easy methods to pull a lookup file from SH to Indexer?

Cheers

0 Karma
1 Solution

somesoni2
SplunkTrust
SplunkTrust

You can just use the summary indexing (saved search fetching data from lookup and saving onto index) to achieve the same. Since, it's a saved search, the SHC will take care of it's execution (once). See this on how to setup a saved search to do summary indexing.

https://docs.splunk.com/Documentation/Splunk/6.5.2/Knowledge/Usesummaryindexing#Set_up_summary_index...

View solution in original post

woodcock
Esteemed Legend

We do this using summary indexing by scheduling a search like this to run daily (or whatever the requirement is):

|inputlookup MyFile.csv | collect index=MySummaryIndex

jkat54
SplunkTrust
SplunkTrust

I would run the same search at the same time but remove the outputlookup command and append the collect command so that it gets collected into an index. Be sure you're forwarding events from the SHC to the indexers as per best practices first.

  ... | collect index=myIndex

koshyk
Super Champion

great option too. thank you

0 Karma

somesoni2
SplunkTrust
SplunkTrust

You can just use the summary indexing (saved search fetching data from lookup and saving onto index) to achieve the same. Since, it's a saved search, the SHC will take care of it's execution (once). See this on how to setup a saved search to do summary indexing.

https://docs.splunk.com/Documentation/Splunk/6.5.2/Knowledge/Usesummaryindexing#Set_up_summary_index...

koshyk
Super Champion

great method. Thanks mate

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...