TA-NMON AIX ERROR ExecProcessor module could not be loaded splunkd.log

[landen99]

How do I resolve the following errors seen on an AIX system running an Universal Forwarder with TA-NMON installed?:

03-26-2017 06:40:40.439 -0400 ERROR ExecProcessor - message from "/opt/splunkforwarder/etc/apps/TA-nmon/bin/nmon_helper.sh" Could not load program /usr/opt/freeware/bin/rpm:
03-26-2017 06:40:40.439 -0400 ERROR ExecProcessor - message from "/opt/splunkforwarder/etc/apps/TA-nmon/bin/nmon_helper.sh" Could not load module /usr/lib/librpm.so.0.0.0.
03-26-2017 06:40:40.439 -0400 ERROR ExecProcessor - message from "/opt/splunkforwarder/etc/apps/TA-nmon/bin/nmon_helper.sh"     Dependent module /opt/splunkforwarder/lib/libz.a(libz.so.1) could not be loaded.
03-26-2017 06:40:40.439 -0400 ERROR ExecProcessor - message from "/opt/splunkforwarder/etc/apps/TA-nmon/bin/nmon_helper.sh"     The module has an invalid magic number.
03-26-2017 06:40:40.439 -0400 ERROR ExecProcessor - message from "/opt/splunkforwarder/etc/apps/TA-nmon/bin/nmon_helper.sh" Could not load module rpm.
03-26-2017 06:40:40.439 -0400 ERROR ExecProcessor - message from "/opt/splunkforwarder/etc/apps/TA-nmon/bin/nmon_helper.sh"     Dependent module /usr/lib/librpm.so.0.0.0 could not be loaded.
03-26-2017 06:40:40.439 -0400 ERROR ExecProcessor - message from "/opt/splunkforwarder/etc/apps/TA-nmon/bin/nmon_helper.sh" Could not load module .
03-26-2017 06:40:40.857 -0400 ERROR ExecProcessor - message from "/opt/splunkforwarder/etc/apps/TA-nmon/bin/nmon_helper.sh" Could not load program /usr/opt/freeware/bin/rpm:
03-26-2017 06:40:40.857 -0400 ERROR ExecProcessor - message from "/opt/splunkforwarder/etc/apps/TA-nmon/bin/nmon_helper.sh" Could not load module /usr/lib/librpm.so.0.0.0.
03-26-2017 06:40:40.857 -0400 ERROR ExecProcessor - message from "/opt/splunkforwarder/etc/apps/TA-nmon/bin/nmon_helper.sh"     Dependent module /opt/splunkforwarder/lib/libz.a(libz.so.1) could not be loaded.
03-26-2017 06:40:40.857 -0400 ERROR ExecProcessor - message from "/opt/splunkforwarder/etc/apps/TA-nmon/bin/nmon_helper.sh"     The module has an invalid magic number.
03-26-2017 06:40:40.857 -0400 ERROR ExecProcessor - message from "/opt/splunkforwarder/etc/apps/TA-nmon/bin/nmon_helper.sh" Could not load module rpm.
03-26-2017 06:40:40.857 -0400 ERROR ExecProcessor - message from "/opt/splunkforwarder/etc/apps/TA-nmon/bin/nmon_helper.sh"     Dependent module /usr/lib/librpm.so.0.0.0 could not be loaded.
03-26-2017 06:40:40.857 -0400 ERROR ExecProcessor - message from "/opt/splunkforwarder/etc/apps/TA-nmon/bin/nmon_helper.sh" Could not load module .

[guilmxm · Accepted solution]

Hi,

This error has been identified, and corrected with the new release 1.3.05 of the TA-nmon which is currently in the end of its qualification.

https://github.com/guilhemmarchand/nmon-for-splunk/tree/testing/resources

You don't say it (in the future think in providing few more elements such as the operating system at least) but this is in relation with IBM AIX systems.

Basically, the solution is to unset the "LIBPATH" before starting topas-nmon, within the function "start_nmon" of the bin/nmon_helper.sh script.

The (new) documentation of the TA-nmon is currently under writing (http://ta-nmon.readthedocs.io/en/testing/)

The new release of the TA-nmon implements a totally reworked processing workflow that uses fifo files, you will be happy to learn that this drastically reduced the CPU / IOPS / Memory overhead, and made being constant and very low.

http://ta-nmon.readthedocs.io/en/testing/footprint.html

The new TA-nmon and the core application have been intensively qualified under dozens of systems, from AIX to Linux x86 and Power architecture, and many issues or bugs have been fixed.

There are as well additional new amazing features, such as the nmon_external which allows you easily integrating any external data within the nmon processing.

The publication of the new release is imminent, most probably within next 2 weeks.

To resume, this new release is focused on drastically improving the back end, and this release will be the best ever 😉

Guilhem

[landen99]

Thank you for the feedback and for the update. The question has been updated to note the AIX connection.

[guilmxm]

Hi,

FYI, the current release v1.3.13 includes the fix for this issue.

Cheers.