Create a search with multiple outputs

[responsys_cm]

I'm trying to figure out if there is some combination of subsearches or other operations that will allow me to accomplish this in a single search. Let's say I'm trying to grab the following statistics from a search on user logins (sourcetype=web_access):

| stats count by username, src_ip | outputlookup lookup1

| stats count by src_ip, browser_agent | outputlookup lookup2

| stats count by username, browser_agent | outputlookup lookup3

Is there a way to have a single root search produce three different sets of stats and output them to a lookup table? I'm trying to minimize the number of scheduled searches I have to manage.

Thx.

C

[johandk]

A single search could do it:

| fillnull username, src_ip, browser_agent value="unknown" | dedup username, src_ip, browser_agent | table username, src_ip, browser_agent | outputlookup lookup1.csv

Your lookup could then work like these:

| lookup lookup1.csv username OUTPUT src_ip, browser_agent
| lookup lookup1.csv src_ip OUTPUT username, browser_agent
| lookup lookup1.csv browser_agent OUTPUT username, src_ip

[sideview]

To back up a second, why not have only one lookup file?

| stats count by username, src_ip, browser_agent | fields - count | outputlookup lookup

In some cases the performance of three separate lookups could be better than one master lookup, but unless your users use tons of IP's and different user_agents, I don't imagine it's the case here.

And I think whatever you have been doing with the three lookups can probably be done with the one lookup.

---

UPDATE: From your comments boiling it down into a single lookup is not an option.

As for what that leaves, the short answer is no you can't really do this.

The long answer is "you can kind of do this, but only in a way that is unspeakably evil".

In a nutshell, Splunk cannot fork search pipelines. It comes up once in a while as a "wouldnt it be cool if", but it's not implemented and I wouldn't bet on it coming in the next few years.

Except...

1) You can abuse the append command to run three completely separate searches, which sort of amounts to the same thing. But DONT do this. For one thing you're running over the exact same data three times. For another thing append has hard limits in the number of rows it can run on, and the number of seconds Splunkd will let the searches run. So unless your three searches are incredibly small and fast, you'll probably be eaten by gremlins.

foo NOT foo | append [foo | stats count by username, src_ip | outputlookup lookup1]
| append [foo | stats count by src_ip, browser_agent | outputlookup lookup2]
| append [foo | stats count by username, browser_agent | outputlookup lookup3]

2) You could save the main search as an alert, with no outputlookup commands, and then have the alert fire always, and run a script. Then in the python script you can use the API or SDK to run the three outputlookup commands as three separate 'postprocess' requests. I think that would work but I'm not sure.

3) go completely off road and use the SDK/API plus cron to do them.

[sideview]

Gotcha. No problem it was worth a shot. 😃 I updated my answer.

[responsys_cm]

In this case I was just using an example. Is it possible to perform three different stats operations on the initial search and write the results to multiple lookup tables?