I need to be able to identify duplicates in a multivalue field. The difficulty is that I want to identify duplicates that match the value of another field.
for example
field1 = "something"
(MV field) field2 = "something, nothing, everything, something"
I need to be able to count how many times field1 is seen.
eval test = mvfilter(match(field2,field1)) <------------- how can I do the equivalent of this since mvfilter doesn't support tokens?
This would likely not be a great search if there are a lot of events that meet your criteria (mvexpand can be a memory hog), but this may work well enough for your use case:
<your base search>
| where field2==field1
| streamstats count AS row
| mvexpand field2
| eval matches=if(field2==field1, field1, dummy)
| stats list(field2) AS field2, values(field1) AS field1, list(matches) AS matches BY row
| table field1 field2 matches
You can try this workaround. (runanywhere search, first two lines are used to generate sample data)
| gentimes start=-1 | eval field1="something" | table field1 | eval field2="something,nothing,everything,something,anything,something,something,something" | makemv field2 delim=","
| eval test=field2 | nomv test| eval test=mvcount(split(replace("start".test."end",field1,"|"),"|"))-1