hi,all,here is my problem:
here is my search:
mysearch | table fields1 fields2
and I got:
fields1 fields2
foofoo abcccd
barbar asdddf
the lookup table I define in lookups is as below,the keywords is regular expression which I want match the fields2
keyword fields3
abccc\w+ 10
asddd\w+ 20
what I want is
fields1 fields2 fields3
foofoo abcccd 10
barbar asdddf 20
so how can I get this done?
I just thought it may be worth pointing out that the mvrex
command which is implemented by the SA-cim_validator app may be something worth taking a look at. While the command itself doesn't deal with lookups, values pulled back from lookups are send through this command on at least one of the dashboards:
https://github.com/hire-vladimir/SA-cim_validator/blob/master/bin/mvrex.py
Anyways, the combo of regex within lookups is pretty rare. Thought this may give some future readers some ideas to think about.
Hi all we have some trouble with this python script
Splunk error code
"returned error code 1"
Please Help
There is no regex support in static lookup tables unfortunately. You could achieve this by writing a dynamic lookup script that does this, the obvious drawback obviously being that it's a bit more hassle to roll up your sleeves and start coding.
I've written this kind of dynamic lookup for this exact purpose and have it lying around somewhere, but don't know where right now - let me know if you want it and I'll have another look.
EDIT: So, looked around and found it. DISCLAIMER, I'm by no means a real Python coder 🙂
#!/usr/bin/python
# A dynamic lookup that takes CSV as input, performs a regex match against another CSV, then returns the CSV results
import csv
import sys
import re
import os
import glob
def inlookup(inf, inval, outf):
try:
# The app makes the assumption that a directory in the form <customer>_indexer_config exists. If multiple
# directories matching this template exist for some weird reason, only the first one is used.
config_app_path = os.path.join(os.environ['SPLUNK_HOME'],'etc','apps','yourapp')
csvname = "yourlookup.csv"
csvpath = os.path.join(config_app_path,'lookups',csvname)
except Exception as e:
sys.stderr.write("No %s file found." % csvname)
sys.exit(0)
try:
c = open(csvpath, 'rb')
f = csv.DictReader(c)
for row in f:
if re.search(row[inf], inval):
return row[outf]
except Exception as e:
sys.stderr.write(e)
sys.exit(1)
return []
def main():
if len(sys.argv) != 3:
print "Usage: %s <in field> <out field>" % (sys.argv[0])
sys.exit(0)
inf = sys.argv[1]
outf = sys.argv[2]
r = csv.DictReader(sys.stdin)
w = csv.DictWriter(sys.stdout, r.fieldnames)
w.writeheader()
for result in r:
# If all fields are already present, there's no need
# to look anything up
if len(result[inf]) and len(result[outf]):
w.writerow(result)
elif len(result[inf]):
outvalue = inlookup(inf, result[inf], outf)
result[outf] = outvalue
w.writerow(result)
main()
As you can see in the start of the inlookup
function you need to specify your path and lookup filename explicitly. As far as I know there's unfortunately no way of providing an argument for a lookup to consume it that way, so it needs to be hardcoded.
transforms.conf
[UniqueID_Lookup]
external_cmd = regexpython.py Id,Name
external_type = python
fields_list = Id,Name
props.conf
LOOKUP-UniqueID_Lookup = UniqueID_Lookup Id AS Id OUTPUTNEW Name AS Name
I want the Name UserDefinedCategory should be displayed in the category...but this is not workin ?? am i missin something ??
Hi Ayn,
Can you pls give me the steps in exucting this ?
I have done the following , but this seems not working
Id is the value that comes in the logs, and correspondingly it matches the Name that are present in the lookup file
ie. if my Id is starting with 2 and ends with 6
thanks,it is very helpful!
Amended my answer with the code I found lying around... 😉
yeah,I really appreciate it if you could have another look,the problem I mentioned is a real case in my work and I stuck here.By the way I write some python script in my daily work,thanks in advance if you could provide the answer!