Getting Data In

Extracting domain names (mvindex?)

howyagoin
Contributor

I get the feeling this is going to be a tough one to solve, but, I'm trying to aggregate results of a search based upon domain name. I realise that this is a bit of a non-starter simply because of things like fubar.com versus fubar.co.uk, but, my first approach to this was:

search term | eval mydomain=split(dest_host,".") | eval tld=mvindex(mydomain,-1) | eval target=mvindex(mydomain,1) | eval hoster=target.".".tld 

And this works most of the time, but not all of the time.

I'm operating on the (questionable) assumption that the last two elements split by dest_host are likely to be the domain name - but maybe there's a better way to perform this aggregation. I'm trying to group together results which might be for host123-ab.fubar.com and host445-qx.fubar.com under fubar.com, for example.

I suppose another way to do this is to use some sort of a lookup table with all well-known TLDs and major sub-domains (.co.uk, .ac.uk and so forth) -- but it feels like a problem others must have tried to resolve here already.

Suggestions welcome!

Tags (2)

woodcock
Esteemed Legend

There are apps for this:

URL Parser: https://splunkbase.splunk.com/app/1545/
URL Toolbox: https://splunkbase.splunk.com/app/2734/

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...