Splunk Search

lookup table populating from a saved search

iKate
Builder

Hello,
can one set up the way how data populates lookup table with results from a saved search: by appending new results or by adding new to the end of file?

It seems that at the moment it overrides all the data in the file each time the search runs by a schedule.
I used thisinstruction

Hope there is a way to add data. Command outputcsv has such option but it writes to a weird destination that can't be changed(?) and the result of outputcsv normally needed in /lookups folder.

0 Karma

MayDayOne
Explorer

Another example for someone to play with. I use this on daily basis to find new URL's my end-users went to.

search x |NOT [inputlookup savedSearch.csv | fields y]   eg. "find x, NOT already listed in your saveSearch.csv"
      | inputlookup savedSearch.csv  append=true         eg. "adds new results to savedSearch.csv"
      | dedup y                                          eg. "removes duplicates"
      | table y                                          eg. "make sure the fields match your original savedSearch.csv fields"
      | outputlookup savedSearch.csv                     eg. "creates new savedSearch.csv"

BenjaminWyatt
Communicator

I believe you just need to add an "append=t" to the inputlookup command. We use the Splunk For Citrix XenApp, and there is a saved search that does exactly this. The syntax is:

search terms | inputlookup myLookup.csv append=t | stats count by InterestingField1, InterestingField2, InterestingField3 | table InterestingField1, InterestingField2, InterestingField3 | output myLookup.csv

This outputs the results to the screen and writes them to the csv. Then you just set up the search to run on a schedule and it builds the lookup for you.

kristian_kolb
Ultra Champion

Benjamin is correct, and you might want to read this, where the process is described in detail.

http://blogs.splunk.com/2011/01/11/maintaining-state-of-the-union/

If you just want to add data, you can probably take it from there.

/k

0 Karma

iKate
Builder

@benjaminwyatt thanks for the suggestion! It lead me to search reference for specifying the command you mentioned 'output' and to my shame only then I found out there's more than one command that can do output. 'outputlookup' can solve my case within the search exactly as you say. But current way of solving this case using configs seems lacks of the option I've asked at first.

0 Karma

BenjaminWyatt
Communicator

So to be clear: you can probably do this by editing savedsearches.conf, but IMHO it's easier to do it within the search syntax itself. One word of caution: your saved search will need to be owned by an account with the appropriate permissions to edit the lookup file. If you're considering playing around with the .conf files, though, access doesn't sound like a problem.

0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

Splunk is officially part of Cisco

Revolutionizing how our customers build resilience across their entire digital footprint.   Splunk ...

Splunk APM & RUM | Planned Maintenance March 26 - March 28, 2024

There will be planned maintenance for Splunk APM and RUM between March 26, 2024 and March 28, 2024 as ...