Matching specific fields in main search with the results from subsearch

[spj2]

I am monitoring a directory with multiple CSV files and indexing these to say an index "ABC". The goal is to extract a field "IP" from this index and match it against multiple fields (src_ip, dst_ip) in different sourcetypes which will be the main search.

I am using the following query:

sourcetype=FW [search index=ABC| rename IP as search | fields search |format]

This query returns events from sourcetype=FW where any field matches "IP" and it's slow. I would like to search the "IP" only in src_ip and dst_ip fields in the FW.

How can I achieve this? Is subsearch the right solution in this case, because the list of "IP" in index "ABC" is going to get longer everyday?

[lguinn2 · Accepted solution]

I am not sure that this will be better, but try it:

sourcetype=FW [ search index=ABC 
      | dedup IP | eval src_ip=IP | eval dest_ip=IP | fields src_ip dest_ip
      | format "(" "(" "OR" ")" "OR" ")" ]

You don't really need all the parentheses in the format command, but it was hard to read when I substituted spaces instead. And the end result will be the same.

It is only one subsearch instead of two, and it will specify the fields to search.

[spj2]

Thanks somesoni2 and lguinn. Both the solutions worked and took exactly the same amount of time.

[somesoni2]

Try This

sourcetype=FW [search index=ABC | stats count by IP | rename IP as src_ip | fields - count] OR [search index=ABC | stats count by IP | rename IP as dst_ip | fields - count]

This will take unique values of IP from index=ABC (stats is the fastest way) then using subsearch a clause will be added as filter "src_ip=". Same is repeated for dst_ip, added as OR clause.

[lguinn2]

What is in the CSV files? Do they contain time-stamped events, or are they more like lists or tables of IP addresses?