Splunk Search

Lookup command using a dynamic csv in a real-time search

thisissplunk
Builder

Does this work? When my lookup table is updated every hour via a separate search, is my real-time search using that newly updated lookup table or not? I understand that if I used my csv via inputlookup in a subsearch that would not be the case. I'm hoping this does work however.

My example:

index=network_logs sourcetype=snort 

Check if the src_ip is the blacklisted ip or not...
| lookup blacklist.csv newest_ip as src_ip OUTPUT newest_ip AS blacklisted_ip_is_src
| fillnull value="" blacklisted_ip_is_src
| eval src_match_or_not=if(blacklisted_ip_is_src != "", "src_matches_hva", "no_match")

Check if the dest_ip is the blacklisted ip or not...
| lookup blacklist.csv newest_ip as dest_ip OUTPUT newest_ip AS blacklisted_ip_is_dest
| fillnull value="" blacklisted_ip_is_dest
| eval dest_match_or_not=if(blacklisted_ip_is_dest != "", "dst_matches_hva", "no_match")

As long as one of them matches, show the event...
| where src_match_or_not="src_matches_hva" OR dest_match_or_not="dst_matches_hva"

report out.....
0 Karma
1 Solution

thisissplunk
Builder

Looks like as long as you are using the lookup command to match things and not something like inputlookup in a subsearch, the "update=true" argument will work as seen here: http://docs.splunk.com/Documentation/Splunk/6.1.4/SearchReference/Lookup

View solution in original post

thisissplunk
Builder

Looks like as long as you are using the lookup command to match things and not something like inputlookup in a subsearch, the "update=true" argument will work as seen here: http://docs.splunk.com/Documentation/Splunk/6.1.4/SearchReference/Lookup

jrodman
Splunk Employee
Splunk Employee

A search using a lookup based on a csv file will use only the form that exists at the start of the execution of the search.

(If the search was to change the lookup definition mid-search, typically customers would get incoherent output from a search.)

You could, if you need to update the lookup dynamically, work around this by using a scripted lookup.

thisissplunk
Builder

This is what I feared. What about the "update" argument for lookup command seen here: http://docs.splunk.com/Documentation/Splunk/6.1.4/SearchReference/Lookup . Would that do it? If not, what does it do?

I need the list to change as soon as the IP addresses for the hosts in question change. So I think this is a legit use of this. If not, can yout hink of some other way to do this with a real-time search? Non real-time searches won't cut it here as we need to ensure we do not miss any events that come in hours late, and need alerts as soon as they hit Splunk.

Lastly, how do I do this scripted lookup thing?

0 Karma

jrodman
Splunk Employee
Splunk Employee

Good eyes. That means my answer is wrong. We do have this feature, for realtime searches specifically, when you would tend to want it.

At least the documentation clearly states that it does what you want. I have not tried it.

0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

Splunk is officially part of Cisco

Revolutionizing how our customers build resilience across their entire digital footprint.   Splunk ...

Splunk APM & RUM | Planned Maintenance March 26 - March 28, 2024

There will be planned maintenance for Splunk APM and RUM between March 26, 2024 and March 28, 2024 as ...