How to correlate DNS values between 2 indexes?

[nts_cseidl]

Dear Splunkers,

I have an index with Windows DNS Logs, where I extract the requested record in to a field --> dns domain.

29.04.2016 08:35:04 16F8 PACKET 0000001680F020F0 UDP Rcv 193.186.217.90 000d Q [0001 D NOERROR] A .cutheatergroup.cn.
29.04.2016 08:31:12 15C4 PACKET 00000016FF0B82A0 UDP Rcv 193.186.217.90 0009 Q [0001 D NOERROR] A .www.subdomain.cutheatergroup.cn.

So in the example above, the dns_domain is:

• cutheatergroup.cn
• subdomain.cutheatergroup.cn

I created another index where I download the malwaredomains.com feed - the log entry looks like this:

29.04.2016 10:02:45 cutheatergroup.cn malware

Here I extract the domain into the field malware_domain and the type into malware_type. In this example:

malware_domain = cutheatergroup.cn
malware_type = malware

Now I want to check if a clients looks up a malware domain. The problem is that the value from dns_domain is not always the same as the value from malware_domain. It's more likely that the malware_domain is a "substring" from the dns_domain or the dns_domain contains the malware_domain.

I tried populating all the values from the malware_domains index with a subsearch and compare is with the value of the dns_domain in the other search. But that is nit working:

index = dns dns_domain= * [search index=security_intelligence sourcetype=security:intelligence:malwaredomains | fields malware_domain,malware_type] | table dns_client malware_domain_full malware_type | eval or where clause to check if there is a match.

Any suggestions for this use case?

Regards

[David]

Absolutely! The URL Toolbox is a great way to extract the actual domain name from the query. Check out: https://splunkbase.splunk.com/app/2734/

For a concrete example of how this works, take a look at this PDF that walks through using URL Toolbox to check entropy of subdomains (toward the end, just search for entropy) with step-by-step examples. You can go through the first few steps to actually extract out the pieces you're looking for.
https://splunk.box.com/v/SplunkLive2016ScottsdaleSec