Splunk Search

Why am I losing a field when dedup-ing values from two sourcetypes?

glenngermiathen
Path Finder

I'm running the following search, but when I add the dedup line my d_name field goes blank. I have two sourcetypes both containing the field d_id. Sourcetype1 has the fields d_id, d_name. Sourcetype2 has the fields d_id, s_id, status. If the dedup line is removed all the fields are populated, but I need to count each s_id once per d_id.

index=d_index
| dedup d_id s_id
| eval S1=mvfilter(match(status, "Open"))
| eval S2=mvfilter(match(status, "Closed"))
| eval S3=mvfilter(match(status, "Ready"))
| stats values(d_name), count(S1) AS Open, count(S2) AS Closed, count(S3) AS Ready by d_id
0 Karma

glenngermiathen
Path Finder

Found the solution. Needed to add this
| join d_id [search sourcetype="source1" | dedup d_id | table d_id d_name]

0 Karma

somesoni2
Revered Legend

Whatever fields you put in dedup should be available in all events. As you stated, it's not available for sourcetype1 hence all the events from sourcetype1 are executed/removed by dedup and d_name is blank. Give this a try

index=d_index
 | eval s_id=coalesce(s_id,"NA")
| dedup d_id s_id
| eval S1=mvfilter(match(status, "Open"))
| eval S2=mvfilter(match(status, "Closed"))
| eval S3=mvfilter(match(status, "Ready"))
| stats values(d_name), count(S1) AS Open, count(S2) AS Closed, count(S3) AS Ready by d_id
0 Karma

glenngermiathen
Path Finder

I downvoted this post because did not give me the d_name, and ended up messing up the counts for the stats fields

0 Karma
Get Updates on the Splunk Community!

Stay Connected: Your Guide to May Tech Talks, Office Hours, and Webinars!

Take a look below to explore our upcoming Community Office Hours, Tech Talks, and Webinars this month. This ...

They're back! Join the SplunkTrust and MVP at .conf24

With our highly anticipated annual conference, .conf, comes the fez-wearers you can trust! The SplunkTrust, as ...

Enterprise Security Content Update (ESCU) | New Releases

Last month, the Splunk Threat Research Team had two releases of new security content via the Enterprise ...