Empty csv lookup file (contains only a header)

[Dimitri_McKay]

I'm getting an error message complaining about a CSV based lookup file containing only a header. But it doesn't. It's chock full of CSV lookup goodness, but my Splunk instance is upset with it for some reason. I'm using release 4.3.4. I attempted to extract the data from the current file to another file in hopes of getting proper results. Thus far? /fail

[iKate]

I faced the same issue. As far as I understood exceeding of default volume of lookup table doesn't impact on the data itself as it just starts to be indexed. So it's enough just to hide this message from a dashboard. To do so add the level field to your Message module:

< module name="Message" layoutPanel="messaging" >
        ...
        < param name="level">error< /param >
< /module >

And it will only emit messages equal to or higher than the specified level.
Splunk's internal logging levels are DEBUG INFO WARN ERROR FATAL (from most to least verbose).

[dbryan]

This it not a problem with your lookup table. As melonman says, it's likely that your lookup has increase to the size where Splunk decides to create an index of the lookup table (if I remember correctly, this is based on the number of rows in the CSV file).

It's quite common that when your lookup file reaches the point where Splunk builds an index it's already more than 10MB. If the max_memtable_bytes is indeed the problem here, that explains why you suddenly see this error message at a seemingly arbitrary point.

In the environment I manage, we're not comfortable setting crazy ceilings on the size of the in-memory index, but nor do we want our lookups to break at random - especially since we let stakeholders upload lookup tables directly.

To get around this, we implemented a scripted lookup called safeLookup that mimics the lookup command.

So this:

`my_search` | lookup myLookupFile.csv

Becomes:

`my_search` | eval lookupName = "myLookupFile.csv" | lookup safeLookup

The scripted lookup can look at the lookupName field to figure out which lookup it's supposed to pretend to be.

Originally this was just in Python, but then we made the Python custom command do a fork and exec to spawn something implemented in native code that does it, and for some use cases than Splunk's native table-based lookup anyway.

Obviously you may not want to do these gymnastics - the point is that there are ways around this issue.

[melonman]

Hi

I got the same message when I tried to lookup fields.

my lookup.csv looks like:

field1,field2,flag
abcdefg1,aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaabbbbbbbbbbbbbbbbbbbbbbcccccccccccccccccccccccccccccccs1,1
abcdefg2,aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaabbbbbbbbbbbbbbbbbbbbbbcccccccccccccccccccccccccccccccs2,1
.. 100,000+ lines ..

When I looked up a field1, it worked fine.
However, I looked up a field2 in the same lookup table, I got the empty csv message.
Is this because of the data lookup gets is very large wit lookup operation with field2..?

so I configured with the following in limits.conf to set to 100MB, and the message went away.

max_memtable_bytes
* Maximum size of static lookup file to use an in-memory index for.
* Defaults to 10000000 in bytes (10MB

I am not sure if this is right way or expected behavior by design...

I hope this helps a bit...

[capo11]

Is this a error in Splunk?

Empty csv lookup file (contains only a header) for table 'ad_old_values': C:\Programmi\Splunk\etc\apps\CentrifyInsight\lookups\ad_old_values.csv

[capo11]

Thank you
Angelo

[jgedeon120]

While I'm not familiar with the Centrify Insight app it looks like the populating search for the lookup has not been ran.

[capo11]

Hi, I have the same problem, when I open the CSV file in Excel I see only header.
Thank you

[jgedeon120]

Capo,

Then there is nothing wrong with your CSV other than there is not data in it other than the headers and has not been populated with data.

[capo11]

I have not changed the format of the csv..
Thank you

objectGUID,uSNChanged,"selected_zone_type","selected_zone_dn","selected_eventtype","ad_old_force_logoff","ad_old_lockout_duration","ad_old_lockout_observation_window","ad_old_lockout_threshold","ad_old_max_pwd_age","ad_old_min_pwd_age","ad_old_min_pwd_length","ad_old_name","ad_old_pwd_history_length","a.....

[jgedeon120]

It looks like you have an issue with the format of your CSV.

[jgedeon120]

If you open the CSV file in a spreadsheet program do you see any issues with the lines/cells?