Using eval to create custom CSV file names and then use that field in lookup as CSV file name

[andra_pietraru]

Hello,

Is it possible to create custom lookup files names and then use them in lookup command in a query?
My events have this field:
... csvfileIndex=1 ...
... csvfileIndex=2 ...
I have many CSV files like: csvfile1.csv, csvfile2.csv. ...

What I would like to achieve is based on that field I would like to lookup a different CSV file. Something like:
... | eval csvName="csvfile".csvfileIndex.".csv" | lookup csvName name AS surname OUTPUTNEW address AS homeaddress | ...
But I am getting error "Error in 'lookup' command: The lookup table 'csvName.csv' does not exist"
Anyone knows what I am doing wrong or if it's even possible to use lookup like this?

[acharlieh]

It'd take a bit of python or scripting work, and I'm not exactly sure about what the script would need to look like to be efficient, but you could potentially write an external lookup for this. There's a generic example about how this works in the docs, and an example script that ships with Sp...

The idea is you have a single external lookup for splunk that takes csvfileIndex and surname and the script upon invocation dynamically retrieves addresses from the appropriate file(s), then your search is easy. (The script to accomplish this however...)

[MichaelPriest]

What are the differences between the files, couldn't you try automatic lookup's instead?

[andra_pietraru]

An automatic lookup would not help, because the content of the csv files is very similar and I would need to lookup only one per event, otherwise my data would get really confusing. That is why I used the csvFileIndex in my events, to make sure I only look at the correct csv file.

[woodcock]

I do not think you can do this but this kind of thing has come up for me several times. What Splunk needs to do is give us a syntax like !value2literal! where we can do something like this:

... | eval csvName="csvfile".csvfileIndex.".csv" | lookup !csvName! name AS surname OUTPUTNEW address AS homeaddress | ..

The problem is that there is no way that I know of to set a Splunk field's value and use it in a place inside a Splunk command's syntax which takes a stirng-literal, such as the lookup stanza name in your example.

[andra_pietraru]

Thanks for the reply! I was hoping it's somehow possible. They should add something like this because it's really inconvenient if you have many CSV files.