Splunk Search

Some keys not being found on lookup table

ibercomm
New Member

Using the UI, I have defined a lookup table from a static CSV file.
In my case, the key/value pairs of the file are as follow:

oui,vendor

0050.56,VMware

0000.09,XEROX

...

Up to 16148 entries (header included).

It all seems to work fine. Most of the lookups return sane results which I have cared to validate. But for some reason, I am missing some hits. 0050.56 will not match against the lookup table, even though it actually is there:

[root@xyz lookups]# pwd
/opt/splunk/etc/apps/search/lookups

[root@xyz lookups]# grep 0050.56 oui.csv
0050.56,VMware

and even though I have properly extracted its value using the "rex" command.

I dont seem to find any logs of the lookups under splunk's var/log, and I dont think I have to escape the "." in "0050.56", as the rest of the keys in the lookup table match without problems.

I'd be very thankful if I could get any pointers on this issue. Splunk is awesome when it works, but when it doesnt... it is not all that easy to debug unless you are very involved with its development or community.

Thanks in advance!

Tags (2)
0 Karma

rschultzsplunk
New Member

I ran into something like this today, also with a csv lookup table for OUIs. In my case, it turned out some of the vendors listed in the wireshark source had double quotes in the name, causing thousands of lines in between the errant quotes to be ignored.

This is an old thread, but in case it helps someone, you can compare the file to what splunk parses out of it. Compare "| inputlookup oui.csv" in splunk (note the leading pipe) with "wc -l oui.csv" in the shell. Splunk will say "0 matching events" but should have a count below listed as "X results". This should be one off of wc's count (due to the header).

You can also try to track down where data is missing by aggregating sections of the file/table and comparing, e.g.:

cut -b 2-2 oui.csv|sort |  uniq -c

vs

| inputlookup oui.csv | rex field=oui "^(?<firstchar>.).:"  | top firstchar limit=16| sort firstchar
0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...