Splunk Search

How to remove duplicates from results of two searches on two different fields that have some of the same values?

BaptVe
Path Finder

Hello !

I launch a search with append to put the results of two searches together on different fields, but then I would like to remove the duplicates on these results:

First LOG :

24/05/2016 11:33:19,719 (...) service id : one

one is the value of the field Service

Second LOG:

24/05/2016 11:38:33,688 (...) service id : two

two is the value of the field state

The two logs are written differently and these two service id have two different field names in Splunk.
I've appended the two results:

index=XXXX com="*xxxx*" service=*  | append [ search
index=XXXX com="*xxxx*"  state=* ] 
| where state!= service |stats list(state)

And I tried with where to show the list, but without success!

Any help is welcome 😄

Thanks !

0 Karma

ishaanshekhar
Communicator

You can modify your search like this...

index=XXXX com="xxxx" service= | rename service as state 
|append [ search index=XXXX com="xxxx" state= ] 
| dedup state |stats list(state)

Thanks
Ishaan

0 Karma

lukasz92
Communicator
  1. Make one field common. For example add to the second search: 'eval service=state'
  2. Use 'dedup service, index, com' (and all other fields that are the same, if there is a duplicate)
0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...