Getting Data In

LookupOperator errors on a forwarder?

Lowell
Super Champion

I've started to see the following messages from some of my forwarding instances of splunk:

11-30-2010 16:50:02.355 ERROR LookupOperator - The lookup table 'sqlagent_jobs' does not exist. It is referenced by configuration 'mssql_processes'.

This error is somewhat correct. I do have "sqlagent_jobs" defined in transforms.conf, but the sqlagent_jobs.csv is missing on the forwarder's app. What I'm trying to figure out is why the forwarder cares.

I'm familiar with getting these LookupOperator errors on the search-head (and interactively) when I forget part of the lookup configuration or have some kind of permissions issue--but I thought that was strictly caused by running a search. On my forwarders, there are no running searches. (I use heavy-weight forwarders, so it's possible to run search on them; but there's no locally indexed data to search; and I see no entries in searches.log or scheduler.log so I don't think that's the case, but I could be missing something.)

Does anyone know what else would trigger this error?

(As a work around, I'm commenting out the LOOKUP-* entries in my deployment-apps app for the moment, but I don't want to have to maintain two versions of my app if I don't have too. I use the same app on the search-head and on my forwarders)

kamal_jagga
Contributor

Check if there was any Windows/OS upgrade/update on the Server.

Restarting the RPC Server of dbconnect and Splunk restart should fix the issue.

0 Karma

kamal_jagga
Contributor

I am also getting the same error on my dbconnect windows server. This started coming over the weekend and no changes were made to any file or config.
Kindly advise.

0 Karma

steveyz
Splunk Employee
Splunk Employee

I'm guessing there are actually searches being run on your heavyweight forwarders. By default I believe we have scheduled searches that run that populate various status dashboards and it may be these searches that are triggering the error.

0 Karma

steveyz
Splunk Employee
Splunk Employee

Hmm, do you ever hit the UI on your forwarders?

0 Karma

Lowell
Super Champion

I think I disabled all the of internal searches for this very reason. But supposing I missed one, can you think of any reasons why I'm not seeing anything logged in searches.log, scheduler.log (both of these are 0 bytes on the forwarder), audit.log has nothing about running searches, and there's nothing in dispatch folder. Is there anything else I should look for?

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...