Why is my dashboard giving error for only one lookup ?

[ramarcsight]

Hello Everyone
I have a dashboard and when i ran it, it gave the following error:

[IDX01] Failed to re-open lookup file: /opt/splunk/var/run/searchpeers/1B9FREERF-50TY-4767-AH6Y-B567EGRYEH-1539009321/kvstore_s_MSS-T@HX1YtUhJKI87IUImcch_netapp_dezbGUbszQGAW242l@NpogZ9AS

[IDX02]Failed to re-open lookup file: /opt/splunk/var/run/searchpeers/1B9FREERF-50TY-4767-AH6Y-B567EGRYEH-1539009321/kvstore_s_MSS-T@HX1YtUhJKI87IUImcch_netapp_dezbGUbszQGAW242l@NpogZ9AS

It gave the error for all the indexers.

original search :

index=foo 
  | lookup device_lookup device_id as id OUTPUT device_ip
  | fields device_ip , orgDeviceName
  | stats count by device_ip 

but when i run

| inputlookup device_lookup

It works fine.

weirdly all other lookups run without local=true. However, only this particular lookup gives error when the search job is run
as

[IDX02]Failed to re-open lookup file: /opt/splunk/var/run/searchpeers/1B9FREERF-50TY-4767-AH6Y-B567EGRYEH-1539009321/kvstore_s_MSS-T@HX1YtUhJKI87IUImcch_netapp_dezbGUbszQGAW242l@NpogZ9AS

Please advise

[dmadeira_splunk]

This might help 

https://docs.splunk.com/Documentation/Splunk/8.1.2/ReleaseNotes/Fixedissues

[Vijeta]

Is it a KV-store lookup ? Are all other lookups similar type or csv lookups?

[ramarcsight]

All are kvstore only

[bishtk]

Hi @ramarcsight , I know its pretty old but just wanted to check if by any chance did you manage to resolve it? As I am facing similar issue now post upgrading environment to 8.1.X

-Thanks

[Laszlo_K]

Hi @bishtk  @ramarcsight 

Had the same "Failed to re-open lookup file" issue with few KVStore lookups.

Found some ".index.lock" artifacts for the lookups having the issue, using this search:

| rest splunk_server=local /servicesNS/-/-/configs/conf-lookups search="*"
| search title="*"
| eval title=if(like(title,"*"),null,title)
| dedup title
| rename eai:acl.app as app, eai:acl.perms.read as read, eai:acl.sharing as sharing
| fields - updated published id eai*
| rename title as stanza
| search stanza="*lock"
| table stanza