Getting Data In

having some trouble with a lookup table.

mloven
Path Finder

Hi all. I'm having some issues getting a lookup table to work properly. Here are the pertinent details:

I have a csv file (called combined.csv) in /opt/splunk/etc/apps/search/lookups that has two columns. It looks like this:

ipaddress,resolved
192.168.1.1,host1
192.168.1.2,host2
192.168.1.3,host3
and so on...

I have this in my /opt/splunk/apps/search/local/transforms.conf:

[dns_lookup]
filename = combined.csv
max_matches = 1

And this in my /opt/splunk/apps/search/local/props.conf:

[dns_lookup]
LOOKUP-dns = dns_lookup ipaddress OUTPUT resolved

I've restarted splunk multiple times.

Basically, I followed the instructions for using a static lookup table here but I'm not getting the results I was shooting for. I was hoping that the "resolved" field would show up when I did a search, but I'm seeing no new fields.

I don't really even know where to begin troubleshooting this one. Anyone have any suggestions?

Tags (3)
0 Karma
1 Solution

tgow
Splunk Employee
Splunk Employee

Make sure that field "ipaddress" is also in your data with a sourcetype of "syslog". The "ipaddress" field with a sourcetype of syslog has to match the first column in your combined.csv file of "ipaddress". Here is an example of what should work:

Sample syslog:
Sep 29 15:57:36 acmepayroll sshd[14867]: Failed password for invalid user rpm from 110.172.158.2 port 33907 ssh2

The "110.172.158.2" must be assigned to the field "ipaddress". If it is not then you can pull the field in the transforms.conf file.

props.conf:

    [syslog]
    LOOKUP-dns = dns_lookup ipaddress OUTPUTNEW resolved
    REPORT-dns = getip

transforms.conf

[dns_lookup]
filename = combined.csv
max_matches = 1
min_matches = 1

[getip]
REGEX = from\s+([^\s]+)
FORMAT = ipaddress::$1

Log back into Splunk and you should be able to run the following search:

sourcetype=syslog | table ipaddress, resolved

View solution in original post

tgow
Splunk Employee
Splunk Employee

Make sure that field "ipaddress" is also in your data with a sourcetype of "syslog". The "ipaddress" field with a sourcetype of syslog has to match the first column in your combined.csv file of "ipaddress". Here is an example of what should work:

Sample syslog:
Sep 29 15:57:36 acmepayroll sshd[14867]: Failed password for invalid user rpm from 110.172.158.2 port 33907 ssh2

The "110.172.158.2" must be assigned to the field "ipaddress". If it is not then you can pull the field in the transforms.conf file.

props.conf:

    [syslog]
    LOOKUP-dns = dns_lookup ipaddress OUTPUTNEW resolved
    REPORT-dns = getip

transforms.conf

[dns_lookup]
filename = combined.csv
max_matches = 1
min_matches = 1

[getip]
REGEX = from\s+([^\s]+)
FORMAT = ipaddress::$1

Log back into Splunk and you should be able to run the following search:

sourcetype=syslog | table ipaddress, resolved

mloven
Path Finder

That was the ticket. I wasn't aware that the column name in my lookup table needed to be a valid field. I changed it to "host" and now it works like a champ.

Thanks all!

0 Karma

hjwang
Contributor

make sure you have the sourcetype dns_lookup, because [stanza_name] is the sourcetype, host, or source to which this lookup applies and can't use regex-type syntax.

mloven
Path Finder

ok, so I changed my props.conf to

[syslog]
LOOKUP-dns = dns_lookup ipaddress OUTPUT resolved

I missed the part where that stanza name had to refer to the sourcetype (or source, host, etc). All of the events on this system are of the syslog sourcetype.

I've restarted again, but I'm still getting the same results.

EDIT... sorry... can't change the formatting in my comment. That props.conf stanza should look exactly like my original one, except the [dns_lookup] has been replaced by [syslog].

0 Karma

gkanapathy
Splunk Employee
Splunk Employee

Well, you can actually use a regex-like syntax for [host::] and [source::] (and also sourcetype stanzas if you really must).

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...