Splunk Search

Compare rows within a table for duplicates

rlautman
Path Finder

Hi,

I have created a report that takes a lookup list of order references and returns all other orders that are related, my problem is that a lot of the results from my search are duplicates - I cannot use a dedup command on any of the fields as I may miss some results - so what I wanted to do was to compare 2 fields within the rows - e.g. OrderID and CompletionStatus - check that there is no row with duplicated information, and remove 1 of the rows if there is. I have seen a similar issue here but this looks like it depends on there being only 2 rows. Is what I am asking possible and if so can anyone suggest how I would go about doing this?

0 Karma
1 Solution

gkanapathy
Splunk Employee
Splunk Employee

I guess I don't see why you can't use

... | dedup OrderID CompletionStatus

which will keep only one of each combination?

View solution in original post

gkanapathy
Splunk Employee
Splunk Employee

I guess I don't see why you can't use

... | dedup OrderID CompletionStatus

which will keep only one of each combination?

rlautman
Path Finder

This seems to do the job, thanks, I wasn't sure if dedup could be used for multiple fields 🙂

0 Karma

kristian_kolb
Ultra Champion

Could the combination of multikv and dedup not be of help?

From the docs on multikv:
Extracts fields from events with information in a tabular format (e.g. top, netstat, ps, ... etc). A new event will be created for each table row. Field names will be derived from the title row of the table.

Then use | dedup field1 field2 field3 to only keep unique combinations of the three fields' values.

/K

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...