Splunk Search

dedup on values of a field that match certain regular expressions

cmak
Contributor

I have a few different values for a Status field that match a certain regular expression that I would like to dedup on.

The following values are possible values for Status:
Active
Resolved *
Closed *

the * indicates wildcard, such as
Resolved (Fixed), Closed (Completed)

I would like a way to dedup on Status so that it can yield a max of 2 events.
Active and either Resolved * or Closed *.

Therefore, if I had 5 events with the following status
Active
Resolved (Fixed)
Resolved (Completed)
Closed (Fixed)
Closed (Completed)

I would like to call dedup on Status field and have only :
Active
Closed (Fixed)

I arbitrarily chose Closed (fixed) as an example to keep. I do not really care, I just would like to keep one of them (i suppose either the earliest or latest one in time would be a good standard).

0 Karma
1 Solution

cmak
Contributor

I have one solution (not sure how good it is) using replace statements

eval simple_status=sStatus|replace Resolved* with Resolved in simple_status| replace Closed* with Resolved in simple_status|dedup simple_status|

View solution in original post

0 Karma

jonuwz
Influencer

This will replace the value of simple_status with the 1st word in the original status

... | rex field=simple_status "^(?<simple_status>\S+).*" | ...

Edit

or if you want Resolved* and Closed* to resolve to the same thing :

... | eval simple_status=if(match(simple_status,"^Resolved.*|^Closed.*"),"Resolved",simple_status)) | ...
0 Karma

jonuwz
Influencer

Sorry - yeah - fixed the typo - and sorry, i misread your question. Alternative solution in answer

0 Karma

cmak
Contributor

Is there a syntax error? It gives me this error:

Error in 'rex' command: The regex '^(<?simple_status>\S+).*' does not extract anything. It should specify at least one named group. Format: (?...).

Also, I would still have two different fields (Closed and Resolved) when I want them to be identical

0 Karma

cmak
Contributor

I have one solution (not sure how good it is) using replace statements

eval simple_status=sStatus|replace Resolved* with Resolved in simple_status| replace Closed* with Resolved in simple_status|dedup simple_status|
0 Karma
Get Updates on the Splunk Community!

More Ways To Control Your Costs With Archived Metrics | Register for Tech Talk

Tuesday, May 14, 2024  |  11AM PT / 2PM ET Register to Attend Join us for this Tech Talk and learn how to ...

.conf24 | Personalize your .conf experience with Learning Paths!

Personalize your .conf24 Experience Learning paths allow you to level up your skill sets and dive deeper ...

Threat Hunting Unlocked: How to Uplevel Your Threat Hunting With the PEAK Framework ...

WATCH NOWAs AI starts tackling low level alerts, it's more critical than ever to uplevel your threat hunting ...