Slow Response DNS Lookup Thousands of IPs

[albyva]

I am running a DNS lookup on IP addresses using the following arrangement, but it is running very, very, very, slow because the initial input is thousands of IPs.

index=generic
| eval minute=strftime(_time, "%H:%M")
| stats sum(bytes) as totalbytes by dest_ip, minute
| eval Mbps=round(totalbytes/1024/1024)
| rename dest_ip as clientip
| lookup dnslookup clientip
| rename clienthost as destination
| table minute, Mbps, clientip, destination
| sort -Mbps
| head 10

Is there anyway this could be rewritten to speed up the Lookup process so that only the Top 10 address are run through Lookup and not the thousands I initially query?

My goal is just to really run the DNS lookup on the Top 10 addresses. But when I move
(head 10) in front of Lookup or Table, it doesn't give me the Top 10 from the Table/Sort.
Maybe this is a Splunk hierarchy problem or maybe I need a whole rewrite.
Any help is appreciated.

[lakromani]

this part

| rename dest_ip as clientip
| lookup dnslookup clientip
| rename clienthost as destination

could be done in one command

| lookup dnslookup clientip as dest_ip output clienthost as destination

[Ayn · Accepted solution]

No, it's because you're moving head before the sort statement. Before you've sorted the events, Splunk will just grab the 10 first entries in whatever order they happen to come in (reverse chronologically). If you do this

index=generic | eval minute=strftime(_time, "%H:%M") | stats sum(bytes) as totalbytes by dest_ip, minute | eval Mbps=round(totalbytes/1024/1024) | sort -Mbps | head 10 | rename dest_ip as clientip | lookup dnslookup clientip | rename clienthost as destination | table minute, Mbps, clientip, destination

You should get the results you want.

[albyva]

w00t. Worked like a charm. Thanks a million