Why do daily outputlookup searches have zero results and leave an empty lookup file?

[pj]

We often create daily lookups from our search results, which are then used for several other key searches. On occasion, upstream processes fail and data that we index into splunk comes in late for whatever reason or doesnt come in at all. The problem is that if our daily outputlookup searches run and pick up zero results they wipe out the lookup file completely and leave us with an empty lookup file.

Looking to do something along these lines:

if event count is less than xxx, then do nothing, 
else
write outputlookup

I am open to other suggestions also to get around this issue. I dont want to append and if the results do come in then I do need to lose the results in the file from the previous day.

Thanks!

[twollenslegel_s]

Another option i was just made aware of that will work today:

This will just create the file if you have > 5000 but less than <10000 events, but can be adjusted to whatever values make sense. 

<populating search>  | eventstats count as table_length
| where table_length > 5000 and table_length < 10000 | fields - table_length | outputlookup create_empty=false lookup.csv

[twollenslegel_s]

Always consider creating an ideas post if you see something you want. 

https://ideas.splunk.com/ideas/EID-I-1474

[woodcock]

Hey, @pj, come back and Pick an answer to Accept to close your question. There are many good answers here, so you might consider an UpVote or 2, also.

[brannonrad]

I had a similar problem - I have some key lookup files that I never want to be empty, because when they become empty, they can wreak havoc on my dashboards! Reasons why they might become empty: low disk space or application issues (i.e., not normal situations). To protect against this, I modified my automated lookup searches as follows:

<search that builds my_lookup.csv> | eval new=1 | append [|inputlookup my_lookup.csv | eval new=0] | eventstats sum(new) as newCount | eval useNum=if(newCount>5,1,0) | where new=useNum | fields - new newCount useNum | outputlookup my_lookup.csv

What I like about my solution is it will exclusively use the new data if it exists and fall back to the old data if the new data has less than 5 events (i.e., there is no "merging" of the new and old data, which can cause issues if data has been deleted.)

[simonverzijl]

Clever solution, took a minute before I got it 😉

[koshyk]

From Splunk 7.1.x , there is a parameter called override_if_empty which does exactly the same (but only when it is empty)

For Previous version 6.x etc. what I have done is two level steps (You could do in one single step if your search is simple). Let's say your old csv is called my_original.csv
1. Write the outputlookup csv based on current data. Call this by another name my_temp.csv
2. Do an append condition and check for valid records on my_temp.csv. If not valid, use the my_original.csv.

Below is an example for checking if there are more than 5 records within the my_temp.csv, otherwise use existing my_original.csv

| inputlookup my_temp.csv
| eval temp="true"
| append [|inputlookup my_original.csv |  eval original="true"]
| eventstats count(temp) as tempCount
| where (tempCount>5 and temp="true") OR ((isnull(tempCount) OR (tempCount<=5)) AND original="true")
| outputlookup my_original.csv

[woodcock]

This will do it, but it necessitates running your search twice:

Your Search Here That Creates "count" | addinfo | eval info_max_time=if((count<xxx), 0, info_max_time) | map search="search earliest=$info_min_time$ latest=$info_max_time$ Your Search Here That Creates "count" | outputlookup SomeFile.csv"

What happens is that passing in 0 for latest causes the search to crash before it gets to the outputlookup command when count < xxx.

[aa70627]

@woodcock that's a creative solution ! Nice