Splunk Search

Why am I losing a field when dedup-ing values from two sourcetypes?

glenngermiathen
Path Finder

I'm running the following search, but when I add the dedup line my d_name field goes blank. I have two sourcetypes both containing the field d_id. Sourcetype1 has the fields d_id, d_name. Sourcetype2 has the fields d_id, s_id, status. If the dedup line is removed all the fields are populated, but I need to count each s_id once per d_id.

index=d_index
| dedup d_id s_id
| eval S1=mvfilter(match(status, "Open"))
| eval S2=mvfilter(match(status, "Closed"))
| eval S3=mvfilter(match(status, "Ready"))
| stats values(d_name), count(S1) AS Open, count(S2) AS Closed, count(S3) AS Ready by d_id
0 Karma

glenngermiathen
Path Finder

Found the solution. Needed to add this
| join d_id [search sourcetype="source1" | dedup d_id | table d_id d_name]

0 Karma

somesoni2
Revered Legend

Whatever fields you put in dedup should be available in all events. As you stated, it's not available for sourcetype1 hence all the events from sourcetype1 are executed/removed by dedup and d_name is blank. Give this a try

index=d_index
 | eval s_id=coalesce(s_id,"NA")
| dedup d_id s_id
| eval S1=mvfilter(match(status, "Open"))
| eval S2=mvfilter(match(status, "Closed"))
| eval S3=mvfilter(match(status, "Ready"))
| stats values(d_name), count(S1) AS Open, count(S2) AS Closed, count(S3) AS Ready by d_id
0 Karma

glenngermiathen
Path Finder

I downvoted this post because did not give me the d_name, and ended up messing up the counts for the stats fields

0 Karma
Get Updates on the Splunk Community!

Updated Team Landing Page in Splunk Observability

We’re making some changes to the team landing page in Splunk Observability, based on your feedback. The ...

New! Splunk Observability Search Enhancements for Splunk APM Services/Traces and ...

Regardless of where you are in Splunk Observability, you can search for relevant APM targets including service ...

Webinar Recap | Revolutionizing IT Operations: The Transformative Power of AI and ML ...

The Transformative Power of AI and ML in Enhancing Observability   In the realm of IT operations, the ...