Deployment Architecture

Can you help me with a query that removes duplicate events?

maheshsat
Explorer

I want to suppress duplicate events. I am trying the below command, but I'm unable to remove duplicate events.

index="id_sp" sourcetype="current" | dedup _raw | search  Period = "Jun-17"  Prepare="Actual" Fixed ="12345"  | rex "(.sap=\"(?[^\",]+))" | table Period  

Note i don't have any fields in which value get changed.

Tags (1)
0 Karma

bjoernjensen
Contributor

Hey,

your table (at the end of your spl) has just Period as column. Since you do a dedup on _raw and search for a certain (no wildcard) Prepare there almost must be only dublicate entries. Moreover your search is optimizable, and maybe you want to add more columns to the table:

index="id_sp" sourcetype="current" Period = "Jun-17" Prepare="Actual" Fixed ="12345"
| dedup _raw
| rex ".sap=(?<sap>\"(?[^\",]+))"
| table Period Prepare Fixed sap

Cheerz,
Björn

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...