Splunk Search

Compare multi value field with an external file

visa87
Explorer

I have a multi value field that looks similar to below
[a,b]
[a,b,c]
[b,c]
.... so on

For each event , I want to compare this with an external file that maintains a list and my final result should be the values that do not match

For eg : my external file contains the following values
a b c d e f

My final result should look like sonething below :
[c,d,e,f]
[d,e,f]
[a,d,e,f]

Can this be achieved in Splunk ?

Tags (1)

sideview
SplunkTrust
SplunkTrust

I'm assuming your external file is a Splunk lookup, or that you can make it a Splunk lookup.
And that your [a,b,c,d,e,f] field is called "foo", and say there's at least one other field in that lookup that is present on all rows, that we'll call "bar".
And also let's assume that in the raw data your multivalue field is called mvField.

<your searchterms> 
| fields mvField (and whatever few other fields you will ultimately need)
| streamstats count as rowId 
| mvexpand mvField 
| lookup foo as mvField OUTPUT bar 
| where isnull(bar)
| stats values(*) as * by rowId

The idea is to use streamstats to paint a unique id on each row, mvexpand out your multivalue field, wash it through a lookup, then discard the rows that actually matched the lookup. Then finally we use stats values(*) to fold all the expanded rows back up, and end up with mvField being a multivalue field again.

0 Karma
Get Updates on the Splunk Community!

Splunk Custom Visualizations App End of Life

The Splunk Custom Visualizations apps End of Life for SimpleXML will reach end of support on Dec 21, 2024, ...

Introducing Splunk Enterprise 9.2

WATCH HERE! Watch this Tech Talk to learn about the latest features and enhancements shipped in the new Splunk ...

Adoption of RUM and APM at Splunk

    Unleash the power of Splunk Observability   Watch Now In this can't miss Tech Talk! The Splunk Growth ...