All Apps and Add-ons

HadoopOps errors & exceptions

hollow
Explorer

I've just installed the HadoopOps app + dependencies and have some problems getting it to work.

First, opening the "Manage Services" popup throws an exception:

 "UnboundLocalError: local variable 'job' referenced before assignment"

Second, the ftr_lookup input does not work. Splunk throws the following exception upon startup:

message from "python /opt/splunk/etc/apps/SA-HadoopOps/bin/scripted_inputs/ftr_lookups.py" splunk.SearchException: Invalid session token: 264afd65bc2120efa7ab3974f42471f8

After running the saved searches referenced in the script all the lookup tables are working.

After fixing the lookup tables, the Home view says:

No matching fields exist

and some searches still don't generate any results ...

Unfortunately the documentation for this app is not very helpful on these issues and there are no (not even unanswered!) questions for HadoopOps in splunkbase. Does anybody even use this app and knows how i can get the errors fixed?

Thanks a lot!

1 Solution

hollow
Explorer

I redid our Hadoop Setup and tried to setup HadoopOps again and found the issue: I didn't have an admin user, so the passAuth stuff did not work. After creating an admin user everything worked out fine.

View solution in original post

0 Karma

hollow
Explorer

I redid our Hadoop Setup and tried to setup HadoopOps again and found the issue: I didn't have an admin user, so the passAuth stuff did not work. After creating an admin user everything worked out fine.

0 Karma

araitz
Splunk Employee
Splunk Employee

Thanks, I have filed the bug HADOOP-1012 to get that changed to splunk-system-user.

0 Karma

rarsan_splunk
Splunk Employee
Splunk Employee

With respect to "Manage Services" modal, it appears the underlying search dispatch for retrieving list of services is failing.
Is it possible you have an older SA-HadoopOps? You can quickly verify by ensuring this file contains the saved search "__retrieve_live_components":
/etc/apps/SA-HadoopOps/default/savedsearches.conf

Then, try to run it separately by going to:
Search App > Click 'Searches & Report' menu > Click '__retrieve_live_components'
where you should expect a list of hosts & services.

If this looks fine, then we need to dig further into what's going on in your specific instance.

Thanks for pointing this out. Irrespective of that search dispatch failure, the exception should be properly handled, and we shall put in appropriate error messaging.

-Roy

rarsan_splunk
Splunk Employee
Splunk Employee

This is due to 'table' command, part of __retrieve_live_components saved search, listing fields that don't exist.

This leads me to believe it's either a field extraction permissions issue, or more likely, the data of source 'ps' is not making it to splunk.

One place to debug further is macro search hadoop_ps to see if hadoop java processes are being captured at all.

0 Karma

hollow
Explorer

the __retrieve_live_components savedsearch exists on my system. running the search manually still gives me "No matching fields exist".

maybe i'm missing some information in my hadoop config that the introspection expects to be there?

0 Karma
Get Updates on the Splunk Community!

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...