Combine multiple saved search job results

[shangshin]

Hi,
Is there a way combine multiple saved search job results?
Something like

| loadjob savedsearch="admin:search:job1" join savedsearch="admin:search:job2"

Thanks in advance!

[rtadams89]

This sounds like a good use of a lookup table. Put whatever search terms you currently have in "job1" in the "..." of the following code

| inputlookup job1results.csv | append [ search ...] | outputlookup job1results.csv

The first time the job runs it will generate an error as the csv file doesn't exist until after the job runs once. After that however, it will just continue to append the results of each successive run to the csv file. At some point, you probably want to either do a "dedup" or completely overwrite the csv so it doesn't get out of hand. You could for example create a second scheduled search that is simply

... | outputlookup job1results.csv

run first thing in the morning, so that job1results.csv only includes results from the searches run that day.

[shangshin]

Wonderful. Thank you!

[rtadams89]

What you'll want to do is run this search every 30 minutes:
| inputlookup job1results.csv | append [ search ...] | search _time>(now()-2592000) outputlookup job1results.csv

This will keep "job1results.csv" updated with all the results that were within the last 30 days.

Then on the dashbaord, use a search like this:

| inputlookup job1results.csv |

[shangshin]

rtadams89 -- thanks for the input.
I have a daily job to run daily stats and it takes 30 minutes to complete. I would like to use a dashboard to show the results for the last 30 days. The sample you show is closed to what I want but it runs search every time. Do you know something like:

| inputlookup job1results.csv | append [ loadjob savedsearch=""] | outputlookup job1results.csv

[shangshin]

My apology if I downvoted your answer but I couldn't find where I downvoted. Please advise and I will correct it.
Your answer is accurate.

[jonuwz]

I'd appreciate it if you could leave a comment on why you downvoted my answer. The question asks how you'd append saved search results - my answer shows exactly how to do that.

[rtadams89]

No, the "outputlookup" command creats the csv file in the lookup directory automatically. "inputlookup" will then find the same csv file in the lookup directory. Like I said, this will cause an error on the first run (as the "inputlookup" comes first, and is trying to load a csv that "outputlookup" hasn't created yet) but after that will work fine.

[shangshin]

Thank you for the advice. Do I need to manually copy over the result.csv to lookup directory?

[rtadams89]

Should also add, there are other ways to get output lookup to append results, instead of using inputlookup at the start. However, doing this way give you the ability to insert a "dedup" command (if appropriate) before the "outputlookup" command.

[jonuwz · Accepted solution]

Join how ?
If there's a common field and you want to join the results :

| loadjob savedsearch="admin:search:job1"
| join common_field [ | loadjsavedsearch="admin:search:job2" ]

Or if you just want the results appended :

| loadjob savedsearch="admin:search:job1"
| append [ | loadjob savedsearch="admin:search:job2" ]

Edit

If you want to concatenate all the previous results of a one particular saved search, the better solution would be to use lookup tables. Using saved search results would be a bad idea because the results eventually expire and get deleted.

[jonuwz]

updated answer

[shangshin]

Thank you. appending the result is what I need.
Assuming job1 runs daily, is it possible to get all the historical results without manuualy typing all the SID ?