Splunk Search

Dedup is not working with mstats

sabari80
Explorer

We are streaming Dynatrace metric data into Splunk, for some reason we are seeing duplicate 'MessageDeduplicationId'. So trying to avoid the duplicate entries using dedup command. But not retrieving any results after using dedup command. Here is my initial query and getting results for this with duplicates-

| mstats sum(calc:service.thaa_stress_requests_count_lr_tags) As "Count" ,avg(calc:service.thaa_stress_requests_lr_tags) As "Response" where index=itsi_im_metrics AND source.name="DT_NonProd_SaaS" by Dimension.id | eval Response=round((Response/1000000),2), Count=round(Count,0) | search Dimension.id IN ("*Process.aspx") After adding dedup to avoid duplicate 'MessageDeduplicationId' , no results | mstats sum(calc:service.thaa_stress_requests_count_lr_tags) As "Count" ,avg(calc:service.thaa_stress_requests_lr_tags) As "Response" where index=itsi_im_metrics AND source.name="DT_NonProd_SaaS" by Dimension.id 
| eval Response=round((Response/1000000),2), Count=round(Count,0) 
| search Dimension.id IN ("*Process.aspx") 
| dedup MessageDeduplicationId sample payload: Dimension.id: xxxProcess.aspx Dimension.name: Literal Not Found MessageDeduplicationId: a901b712889217fc194cd0446a70325e aggregation: avg entity.service.id: xxx entity.service.name:xxxx metric_name:calc:service.thaa_stress_requests_lr_tags: 1613759 resolution: 1m source.name: xxxx unit: MicroSecond
Labels (2)
Tags (1)
0 Karma

sabari80
Explorer

sample Payload

=========

 

sample payload:

Dimension.id: xxxProcess.aspx

Dimension.name: Literal Not Found

MessageDeduplicationId: a901b712889217fc194cd0446a70325e

aggregation: avg

entity.service.id: xxx

entity.service.name:xxxx

metric_name:calc:xxxx_

lr_tags: 1613759

resolution: 1m s

ource.name: xxxx

unit: MicroSecond
0 Karma

sabari80
Explorer

Modified Query

==========

 

| mstats sum(calc:service.thaa_stress_requests_count_lr_tags) As "Count" ,avg(calc:service.thaa_stress_requests_lr_tags) As "Response" where index=itsi_im_metrics AND source.name="DT_NonProd_SaaS" by Dimension.id | eval Response=round((Response/1000000),2), Count=round(Count,0) | search Dimension.id IN ("*Process.aspx") | dedup MessageDeduplicationId 
0 Karma

sabari80
Explorer

Initial Query

==========

 

| mstats sum(calc:service.thaa_stress_requests_count_lr_tags) As "Count" ,avg(calc:service.thaa_stress_requests_lr_tags) As "Response" where index=itsi_im_metrics AND source.name="DT_NonProd_SaaS" by Dimension.id | eval Response=round((Response/1000000),2), Count=round(Count,0) | search Dimension.id IN ("*Process.aspx")
0 Karma
Get Updates on the Splunk Community!

Modern way of developing distributed application using OTel

Recently, I had the opportunity to work on a complex microservice using Spring boot and Quarkus to develop a ...

Enterprise Security Content Update (ESCU) | New Releases

Last month, the Splunk Threat Research Team had 3 releases of new security content via the Enterprise Security ...

Archived Metrics Now Available for APAC and EMEA realms

We’re excited to announce the launch of Archived Metrics in Splunk Infrastructure Monitoring for our customers ...