Splunk Search

How to filter out all events that have duplicate values from my search results?

tnoelOTS
Explorer

I am running a search of my Rapid7 data I need to compare 2 fields Dest_ip and signature_id If both fields have the same data I want to remove all records that have that data from my search.

Example

event 1: dest_ip=10.10.10.10 signature_id=1
event 2: dest_ip=10.10.10.10 signature_id=1
event 3: dest_ip=10.10.10.10 signature_id=2

results after search would only give me the unique value for event 3

0 Karma

sundareshr
Legend

Try the dedup command

base search | dedup dest_ip signature_id

https://docs.splunk.com/Documentation/Splunk/6.5.0/SearchReference/Dedup

UPDATED

base search | eventstats count by dest_ip signature_id | where count=1

tnoelOTS
Explorer

The Dedup command is not working for this application because it returns 1 of the results that had a duplicate value so in my example above dedup gives me Event 1 and event 3 I want to only get event 3 from the search results.

0 Karma

aaraneta_splunk
Splunk Employee
Splunk Employee

@tnoelOTS - Did sundareshr's updated search provide a working solution to your question? If yes, please don't forget to resolve this post by clicking "Accept". If no, please leave a comment with more feedback. Thanks!

0 Karma

sundareshr
Legend

Try the updated search

0 Karma

gokadroid
Motivator

Updated search should work.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...