Getting Data In

LookupOperator errors on a forwarder?

Lowell
Super Champion

I've started to see the following messages from some of my forwarding instances of splunk:

11-30-2010 16:50:02.355 ERROR LookupOperator - The lookup table 'sqlagent_jobs' does not exist. It is referenced by configuration 'mssql_processes'.

This error is somewhat correct. I do have "sqlagent_jobs" defined in transforms.conf, but the sqlagent_jobs.csv is missing on the forwarder's app. What I'm trying to figure out is why the forwarder cares.

I'm familiar with getting these LookupOperator errors on the search-head (and interactively) when I forget part of the lookup configuration or have some kind of permissions issue--but I thought that was strictly caused by running a search. On my forwarders, there are no running searches. (I use heavy-weight forwarders, so it's possible to run search on them; but there's no locally indexed data to search; and I see no entries in searches.log or scheduler.log so I don't think that's the case, but I could be missing something.)

Does anyone know what else would trigger this error?

(As a work around, I'm commenting out the LOOKUP-* entries in my deployment-apps app for the moment, but I don't want to have to maintain two versions of my app if I don't have too. I use the same app on the search-head and on my forwarders)

kamal_jagga
Contributor

Check if there was any Windows/OS upgrade/update on the Server.

Restarting the RPC Server of dbconnect and Splunk restart should fix the issue.

0 Karma

kamal_jagga
Contributor

I am also getting the same error on my dbconnect windows server. This started coming over the weekend and no changes were made to any file or config.
Kindly advise.

0 Karma

steveyz
Splunk Employee
Splunk Employee

I'm guessing there are actually searches being run on your heavyweight forwarders. By default I believe we have scheduled searches that run that populate various status dashboards and it may be these searches that are triggering the error.

0 Karma

steveyz
Splunk Employee
Splunk Employee

Hmm, do you ever hit the UI on your forwarders?

0 Karma

Lowell
Super Champion

I think I disabled all the of internal searches for this very reason. But supposing I missed one, can you think of any reasons why I'm not seeing anything logged in searches.log, scheduler.log (both of these are 0 bytes on the forwarder), audit.log has nothing about running searches, and there's nothing in dispatch folder. Is there anything else I should look for?

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...