DNS Lookup via Splunk

[Hazel]

Hello,

I am trying to use the external_lookup.py feature to pass in IP addresses and return the hostname.

I tried copying the files from here http://www.splunk.com/base/Documentation/4.1.5/Knowledge/Addfieldsfromexternaldatasources

So i have transforms:

[dnsLookup]
external_cmd = external_lookup.py host ip
fields_list = host, ip

For a reverse DNS lookup, your props.conf stanza would be:

[access_combined]
lookup_rdns = external_lookup.py ip AS clientip OUTPUTNEW host AS hostname

But when I try and run it I just get an error saying that dnsLookup does not exist. Can you tell me what I am doing wrong? These two files don't look quite right to me, but I am following the guide on the splunk page

Thanks Hazel

[araitz]

DNS lookups, especially reverse, are expensive and generally sub-optimal, so that might be the thing you are doing wrong 😛

I recommend creating a temporal/time-based lookup from sources like DNS, DHCP, WinEventLog:Security, etc and using that to decorate your events.

Aside from offloading the DNS queries and giving you multiple sources for corroboration, you can also harvest machine name, NetBIOS name, and other interesting artifacts such as authentication events.

[chris · Accepted solution]

Hi Hazel

You are right, I think the guide is not quite correct, the "external_lookup.py" should be "dnsLookup" in props.conf. But don't worry read on.

To get started the definition in transforms is enough. The props.conf will only do the lookup automatically on the sourcetype/source/host you specify

So if you have the following in transforms.conf

[dnsLookup]
external_cmd = external_lookup.py host ip
fields_list = host, ip

Then you can do forward lookups by using yoursearch | lookup dnsLookup host

If you have a different field that you want to use as the hostfield yoursearch | lookup dnsLookup host as my_own_field

To do reverse lookups it is the same you just need to have a field with an ip-address, again add the as my_own_fieldname if the field is not called "ip"

yoursearch | lookup dnsLookup ip
yoursearch | lookup dnsLookup ip as my_own_fieldname

Now if you want to have reverse lookups done automatically this is a (not exactly perfect) example

This is a sample sendmail event:

Apr  6 17:08:38 splunk3 sendmail[10153]: n36N8bTs010153: from=<0403pc@163.com>, size=0, class=0, nrcpts=0, proto=SMTP, daemon=MTA, relay=61-231-68-166.dynamic.hinet.net [61.231.68.166]

I want to extract the IP-Adress in the brackets and do a reverse lookup on that. This is my props.conf:

[sendmail]
EXTRACT-bla = relay=.*\[(?<myip>.+)\]
lookup_rdns = dnsLookup ip as myip  OUTPUT host AS host_rev_lookup

I hope this helps

Chris

[justin0104]

I'm trying do a reverse DNS lookup in splunk and i'm also having some issues. I have the following in my props.conf file...

[access_combined]
LOOKUP-rdns = dnsLookup ip AS clientip OUTPUTNEW host AS hostname

I think the problem may be that the field that i'm trying to do the reverse lookup on isn't called ip or clientip, it's called client_ip. My transforms.conf looks like this:

[dnsLookup]
external_cmd = external_lookup.py host ip
fields_list = host, ip

Just looking for some direction here. I've been unable to get a reverse lookup to work so far. Thanks

[yannK]

Remark : this answer was got an older version of splunk, the lookup definition by default in splunk (6.2.* now)

The lookup already exists in splunk :
dnslookup with the fields clientip and clienthost
manager > lookup > defintions

To use the lookup on the search
example :
sourcetype=win* |stats count by src_ip | lookup dnslookup clientip As src_ip OUTPUT clienthost

Remarks :
- Do not make it an automatic lookup, as it is not optimized, and it does not need to apply to each events. Instead, use the lookup at the end of the search when you already have grouped your events per ip.
- If you really want to make an automatic lookup, use the manager UI, and pick the dnslookup form the list .
- List item

[wsnyder2]

It is awesome that they built this into version 6.2+!

I tried it, but with no luck. What I would to get is the IP address of the Splunk UF associated with an event.
We have a ton of events coming from cloud hosts which have artifical hostnames that we have created and assigned in the inputs.conf. So I can't use "host" value in splunk for a reverse lookup.

What I really want to reveal is the ip associated with the Splunk UF agent for each event.
I can see these ip addresses in the Deployment Server view but they are lost as my search head view.
Any suggestions?

[johnnybravo]

Ok so I see that it is working. I can see a hostname field listed on the left under other interesting fields. I want Splunk to replace the IP address in the host field with the rDNS name. So that I can search based on host name and so that the hostnames show up in the actual syslog entries. Please help?

[johnnybravo]

I am not able to get this to work. I can run the commands manually without error but I do not see any change. What I want to do is show the host IP as the host name (via reverse DNS).

I have this in transforms.conf

[dnsLookup]

external_cmd = external_lookup.py host ip

fields_list = host, ip

I have this in props.conf

[syslog]

lookup-rdns = dnsLookup ip AS host OUTPUT host as hostname

I tried changing the hostname at the end to host, but no change. It seems the problem is that hostname doesn't get used in the output of my syslog files?

[aferone]

This is the best answer I have found on this subject. I got mine working as well. Thank you, Chris!!

[Hazel]

Hi thanks for this. Actually, I commented out my props.conf and matched my transforms to you and it picked up the lookup table so it looks like the props was messing it up :). All good now, working great!