Splunk Dev

Why does my external scripted lookup give me an error code 1?

reswob4
Builder

So I created a way to use Mark Baggett's freq_server script as a lookup and blogged about it here (http://shadowtrackers.net/blog/get-your-freq-on-in-splunk).

But, that is not the point of this forum post.

I had based my blog post on my setup at home. I was running a single Splunk server and making the URL connection to a separate Linux server running in a vm on the same box as the Splunk server. So, when I got to work and tried to implement the same lookup, I was surprised and frustrated when it didn't work right out of the box. On the small Splunk instance where I implemented the lookup, I have a single search head and two Indexers. When I ran the lookup search:

index=dns | rename query as domain | lookup freqserver domain

I received the following error:

    2 errors occurred while the search was executing. Therefore, search results might be incomplete. Hide errors.   

        [INDEX01] Script for lookup table 'freqserver' returned error code 1. Results may be incorrect.
        [INDEX02] Script for lookup table 'freqserver' returned error code 1. Results may be incorrect.
0 Karma
1 Solution

reswob4
Builder

My first thought was WTH?

But here are some of the things I tried:

https://answers.splunk.com/answers/369810/python-alert-script-fails-and-i-cant-see-errors-in.html
https://answers.splunk.com/answers/559456/script-for-lookup-table-whois-returned-error-code.html

But ultimately the information in the error clued me in. The linux server at work where I was running the frequency analysis server had iptables running and I had configured it to allow my SH to connect on the exact port the frequency analysis server was listening on. But what I didn't know, and isn't mentioned here (https://docs.splunk.com/Documentation/Splunk/latest/Knowledge/Configureexternallookups), is that the script is actually run from the search peers (which in my case were the indexers). Since the search peers were not permitted to connect to the frequency analysis server per the iptables rules, the script was blocked from connecting and thus it failed.

Once I opened up iptables to allow the search peers access to the frequency analysis server, everything worked.

Maybe someone from Splunk can better explain, but apparently the script is copied from the SH to one of the following locations (or both) on each search peer and run from there:

$SPLUNK_HOME$/var/run/searchpeers/<servername>-#########/searchscripts/

or

$SPLUNK_HOME$/var/run/searchpeers/<servername>-#########/system/bin/

Hopefully this helps someone else troubleshoot their scripted lookups....

View solution in original post

0 Karma

reswob4
Builder

My first thought was WTH?

But here are some of the things I tried:

https://answers.splunk.com/answers/369810/python-alert-script-fails-and-i-cant-see-errors-in.html
https://answers.splunk.com/answers/559456/script-for-lookup-table-whois-returned-error-code.html

But ultimately the information in the error clued me in. The linux server at work where I was running the frequency analysis server had iptables running and I had configured it to allow my SH to connect on the exact port the frequency analysis server was listening on. But what I didn't know, and isn't mentioned here (https://docs.splunk.com/Documentation/Splunk/latest/Knowledge/Configureexternallookups), is that the script is actually run from the search peers (which in my case were the indexers). Since the search peers were not permitted to connect to the frequency analysis server per the iptables rules, the script was blocked from connecting and thus it failed.

Once I opened up iptables to allow the search peers access to the frequency analysis server, everything worked.

Maybe someone from Splunk can better explain, but apparently the script is copied from the SH to one of the following locations (or both) on each search peer and run from there:

$SPLUNK_HOME$/var/run/searchpeers/<servername>-#########/searchscripts/

or

$SPLUNK_HOME$/var/run/searchpeers/<servername>-#########/system/bin/

Hopefully this helps someone else troubleshoot their scripted lookups....

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...