Splunk Search

Why am I losing a field when dedup-ing values from two sourcetypes?

glenngermiathen
Path Finder

I'm running the following search, but when I add the dedup line my d_name field goes blank. I have two sourcetypes both containing the field d_id. Sourcetype1 has the fields d_id, d_name. Sourcetype2 has the fields d_id, s_id, status. If the dedup line is removed all the fields are populated, but I need to count each s_id once per d_id.

index=d_index
| dedup d_id s_id
| eval S1=mvfilter(match(status, "Open"))
| eval S2=mvfilter(match(status, "Closed"))
| eval S3=mvfilter(match(status, "Ready"))
| stats values(d_name), count(S1) AS Open, count(S2) AS Closed, count(S3) AS Ready by d_id
0 Karma

glenngermiathen
Path Finder

Found the solution. Needed to add this
| join d_id [search sourcetype="source1" | dedup d_id | table d_id d_name]

0 Karma

somesoni2
Revered Legend

Whatever fields you put in dedup should be available in all events. As you stated, it's not available for sourcetype1 hence all the events from sourcetype1 are executed/removed by dedup and d_name is blank. Give this a try

index=d_index
 | eval s_id=coalesce(s_id,"NA")
| dedup d_id s_id
| eval S1=mvfilter(match(status, "Open"))
| eval S2=mvfilter(match(status, "Closed"))
| eval S3=mvfilter(match(status, "Ready"))
| stats values(d_name), count(S1) AS Open, count(S2) AS Closed, count(S3) AS Ready by d_id
0 Karma

glenngermiathen
Path Finder

I downvoted this post because did not give me the d_name, and ended up messing up the counts for the stats fields

0 Karma
Get Updates on the Splunk Community!

Announcing Scheduled Export GA for Dashboard Studio

We're excited to announce the general availability of Scheduled Export for Dashboard Studio. Starting in ...

Extending Observability Content to Splunk Cloud

Watch Now!   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to leverage ...

More Control Over Your Monitoring Costs with Archived Metrics GA in US-AWS!

What if there was a way you could keep all the metrics data you need while saving on storage costs?This is now ...