Splunk can't write index file for lookup table

[responsys_cm]

I've recently started getting the following error when running a search that previously was working:

Empty csv lookup file (contains only a header) for table 'nessus_plugin_reference_lookup': C:\Program Files\Splunk\etc\apps\ResponsysSecurityConsole\lookups\nessus_plugin_reference_lookup.csv

I see the following errors in Splunkd:

10-17-2012 14:09:03.796 -0700 WARN ProcessDispatchedSearch - PROCESS_SEARCH print_containing_directory_contents - Unable to create file "C:\Program Files\Splunk\etc\apps\ResponsysSecurityConsole\lookups\nessus_plugin_reference_lookup.csv.index\0-0-26615912336424.pre-tsidx\current_time.file" (errno=3).

10-17-2012 14:09:03.796 -0700 WARN ProcessDispatchedSearch - PROCESS_SEARCH print_containing_directory_contents - Dumping contents of dir that contains file="C:\Program Files\Splunk\etc\apps\ResponsysSecurityConsole\lookups\nessus_plugin_reference_lookup.csv.index\0-0-26615912336424.pre-tsidx" time(NULL)=1350508143

How do I diagnose this?

Thx.

Craig

[responsys_cm]

From Splunk support:

Hello Craig,

I believe you will need to increase a setting in limits.conf in order to properly deal with this larger lookup table.

The parameter in question is max_memtable_bytes that is set by default to 10MB

In $splunk_home/etc/system/local/limits.conf you will need to increase the value as so:

[lookup]
max_memtable_bytes = 50000000

To account for your larger lookup file. Once this is set and the instance restarted, the lookup should perform properly. More information on the limits.conf file can be found here:

http://docs.splunk.com/Documentation/Splunk/5.0/admin/Limitsconf

Here is an answers post related to the issue:

http://splunk-base.splunk.com/answers/8326/are-lookup-tables-indexed

[bmacias84]

Increase the Logging channel for ProcessDispatchSearch. Might provide additional information.