Splunk Search

How do you lookup match field names by wildcard or regex?

xshen_anji
New Member

I have some customer provided CSV lookup files. These lookup files have some "similar" field names, which means they contain some common keywords. I would like do a keyword match in lookup command to these similar fields. Is there a way to do this ?

An example is:

lookup file1's title is like: population, average income, location
lookup file2's title is like: population, income, location

I would like to use the field that contains "income" as the lookup condition, how can I do this with one common lookup statement?

Tags (3)
0 Karma

woodcock
Esteemed Legend

You are misunderstanding the 2 different ways to use lookup files. One way is with the ... | lookup command syntax, which uses the WILDCARD() syntax (among other settings) within the Lookup definitions, the other is with the |inputlookup command syntax which DOES NOT interact with the Lookup definitions. In the latter case, just do something like this:

index=_internal [ |inputlookup hosts_reporting.csv | eval host=host + "*" ]

ddrillic
Ultra Champion

Very interesting @woodcock - thank you for the clarification.

0 Karma

woodcock
Esteemed Legend

If you have your answer, pick one and click Accept to close the question.

ddrillic
Ultra Champion

Hi @woodcock - it's not mine - I just joined the ride ; -)

0 Karma

woodcock
Esteemed Legend

ddrillic
Ultra Champion

According to How to use wildcard in lookup-based searches and alerts?

You can specify -

 match_type = WILDCARD(income)

In the transforms.conf definition of your lookup.

I just tested it, I have -

$SPLUNK_HOME/etc/apps/search/local

$ cat transforms.conf 

[hosts_reporting]
batch_index_query = 0
case_sensitive_match = 1
filename = hosts_reporting.csv
match_type = WILDCARD(host)

One of the host names in hosts_reporting.csv is the beginning of a host name and it comes up via -

index=_internal [ | inputlookup hosts_reporting.csv | eval host=host + "*" ]

But it doesn't when running -

index=_internal [ | inputlookup hosts_reporting ]

Isn't it weird?

0 Karma

ddrillic
Ultra Champion

Any thoughts on this one, by any chance? @woodcock?

0 Karma

somesoni2
SplunkTrust
SplunkTrust

How are you planning to run lookup on both lookup table files? What all have you tried so far? Any specific reason to have common lookup statement?

0 Karma

xshen_anji
New Member

Basically, I am running an app which processes a lookup file with some data files. The lookup csv file, which is generated from reporting systems of different vendors, varied a little bit in title fields, but the keywords are basically the same . Since there are a lot of customers, it would be easy to manage if I have one lookup statement to support all these lookup files. Now I am trying to make it a rule to all the customers, asking for manually editing the titles to make them the same, but it would still be desirable to tolerate some kind of fault or difference.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...