Splunk Search

Having a search trigger another search

kholleran
Communicator

Is there a way to trigger another search from a search? What I have is a syslog search for traffic on a router. The only way to connect to that router is a windows server. If the first search yields results, I would like those results emailed. I have this working.

I also would like a search that tells me successful logins to the windows server in a given time period. This works fine.

However, what I want to do is if the first search finds anything, then trigger the second search. I have tried ways to do this with subsearches as well as appending the two searches, which works but has too much data (I just want the source IP and a few other fields from the syslog traffic and the username from the server login, not a list of the raw event data).

I also tried to set up the second search as an alert with the customer alert condition being the first search, which also did not work.

The only thing I can think of is to script the second search and have the first search trigger that script. I am familiar with Python but not very much with Splunk.

Is there another way to do this and if not, can someone give me any guidance on how to script this?

Thanks.

Kevin

Tags (1)

kamal_jagga
Contributor

You need to use shell script and curl command to achieve this.

  1. In the first alert, put the alert condition and run a shell script from that.
  2. In the shell script use the curl command to trigger the 2nd alert.
0 Karma

Lowell
Super Champion

I've been working on a solution to do this (link below). As of yet, it not working 100% and there are some gotchas. (One of the bugs preventing me from doing this was schedule to be fixed in 4.1.4, but I haven't gotten around to verifying that it resolved the issue for me.) The other issue has to do with missing UI support, but I suspect that could be some work around for this too (but for now it means editing savedsearches.conf by hand.)

I'll try to update this (and possibly publish it in the form of an "app") once I get something that's worth sharing. But the basic premise of my solution is to simply add a new alert action that simply launches another saved search. (The approach is very simple at the moment, but I have some other features I'd like to play around with. For example, passing a 'pid' (or some other field) form the first search to the second search using a search template.)

Related question:


If you would like to work together on a solution. Please let me know. I can send you what I've come up with so far as a starting point.


Update: I have confirmed that 4.1.4 resolved the technical issue that was preventing my solution from working. So I should be able to release a working app for this in the near future; assuming I get enough spare time to complete this.


I have posted an my app on SplunkBase, which you can download here:

Note: This app was uploaded Aug 5th and is currently pending approval.

jnhth
Explorer

Does this app still exist?

0 Karma

isoutamo
SplunkTrust
SplunkTrust
If you need the functionality that do a search only if first search return something then you should look require command on splunk 8.2. This is probably what you are needing?
https://docs.splunk.com/Documentation/Splunk/8.2.2/SearchReference/Require
r. Ismo
0 Karma

Genti
Splunk Employee
Splunk Employee

(i assume) sourceip and username are actually fields. If so then i think you can use:

< your search > | fields sourceip, username | dedup sourceip

Try the above in the search that runs second, this way perhaps you will get less data and only the fields you want?
Note, i havent tested this so i cant tell if it works 100% but i think it should...

Best,
.gz

0 Karma

Genti
Splunk Employee
Splunk Employee

(sorry, couldnt fit all the reply into one comment)

0 Karma

Genti
Splunk Employee
Splunk Employee

So you can ask your search to only show you the field you are interested in, and dedup by that field (so you do not get duplicates)...
Perhaps im not understanding your question too well either 😕

0 Karma

Genti
Splunk Employee
Splunk Employee

well from what i understood it seemed like the first search is running, and returning some sort of result. If the result > 0 (or whatever condition you have put) it seems like a second search is running and returning results, but theres too much noise and not easy for you to see the necessary info. (you say - "I have tried ways to do this with subsearches as well as appending the two searches, which works but has too much data")
So, if that is the case, what you see is results to your second search but too much noise,

0 Karma

kholleran
Communicator

Thank you for the response. The two searches do not have any related fields. The first search is syslog data which has items like host, src_ip, and some custom fields to provide information about the location. The second is hitting the windows event log, which I only care about the Account_Name field .

I guess I do not fully understand your suggestion.

Thanks.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...