Splunk Search

Is it possible to remove values specified in a field from a list of values in another field?

andrewtrobec
Motivator

Hello,

Given two list fields add and remove, as well as a currentList list field, is it possible to create a new fieldupdatedListso that gets updated with add values and without remove values?

add,remove,currentList,updatedList
value1,value2,value2 value3 value4, value1 value3 value4
value5,value3 value4,value1 value3 value4,value1 value5

Any hints would be great!

Best regards,

Andrew

Tags (1)
0 Karma
1 Solution

micahkemp
Champion

Here's a run-anywhere possibility:

| makeresults | eval add="value1", remove="value2", current="value2 value3 value4"
| append [| makeresults | eval add="value5", remove="value3 value4", current="value1 value3 value4"]
| makemv add 
| makemv remove 
| makemv current
| eval current_plus_add=mvappend(add, current)
| streamstats window=1 current=t values(current_plus_add) AS distinct_current_plus_add, values(remove) AS distinct_remove
| eval distinct_current_plus_add_plus_distinct_remove=mvappend(distinct_current_plus_add, distinct_remove)
| streamstats count AS serial
| stats count BY distinct_current_plus_add_plus_distinct_remove serial
| search count=1
| stats list(distinct_current_plus_add_plus_distinct_remove) AS after_add_remove BY serial

View solution in original post

micahkemp
Champion

Here's a run-anywhere possibility:

| makeresults | eval add="value1", remove="value2", current="value2 value3 value4"
| append [| makeresults | eval add="value5", remove="value3 value4", current="value1 value3 value4"]
| makemv add 
| makemv remove 
| makemv current
| eval current_plus_add=mvappend(add, current)
| streamstats window=1 current=t values(current_plus_add) AS distinct_current_plus_add, values(remove) AS distinct_remove
| eval distinct_current_plus_add_plus_distinct_remove=mvappend(distinct_current_plus_add, distinct_remove)
| streamstats count AS serial
| stats count BY distinct_current_plus_add_plus_distinct_remove serial
| search count=1
| stats list(distinct_current_plus_add_plus_distinct_remove) AS after_add_remove BY serial

andrewtrobec
Motivator

Beautiful! I hope that a future feature for multivalues will be mvremove or something like that so we can easily remove values instead of tricky workarounds!

0 Karma

niketn
Legend

@andrewtrobec, you can create your own custom commands 🙂

https://docs.splunk.com/Documentation/Splunk/latest/Search/Customsearchcommandshape
http://dev.splunk.com/view/python-sdk/SP-CAAAEU2

____________________________________________
| makeresults | eval message= "Happy Splunking!!!"
0 Karma

somesoni2
SplunkTrust
SplunkTrust

Is field currentList multivalued field? Also, are add and remove multivalued field or single valued?

andrewtrobec
Motivator

currentList, add, and remove are all multivalue fields. I've figured the add part out:

| eval currentList=mvdedup(mvappend(currentList, add))

I just need to figure out the remove part.

0 Karma

somesoni2
SplunkTrust
SplunkTrust

Unfortunately, there is no easy/direct way for removal from multivalued field. One method could be like this.

your current search with fields add, remove and currentList
| eval currentList=mvdedup(mvappend(currentList, add))
| nomv remove 
| streamstats count as sno
| mvexpand currentList
| where len(replace(remove,currentList,"")=len(remove)
| stats values(*) as * by sno | fields - sno
0 Karma

andrewtrobec
Motivator

I've cut and pasted the code into my search but it doesn't seem to work, it just kinda messes up the fields. I'll deconstruct the logic and play around with it. Thanks!

0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

Splunk is officially part of Cisco

Revolutionizing how our customers build resilience across their entire digital footprint.   Splunk ...

Splunk APM & RUM | Planned Maintenance March 26 - March 28, 2024

There will be planned maintenance for Splunk APM and RUM between March 26, 2024 and March 28, 2024 as ...