All Apps and Add-ons

ModularInput Can't connect to Splunk REST API token is invalid or SplunkD has exited HTTP 401

simony
Path Finder

Hi Splunkers

I have a problem with a Modular Input in Splunk. I'm using the Monitoring of Java Virtual Machines with JMX app to push JMX data into Splunk.
This works with a new Installation from a heavy forwarder very well. After a time, the Java process exits with the following message:

 ERROR ModularInput - Can't connect to Splunk REST API with the token [Splunk bgmFr7ozce8o6MsM_XSjtBctMT62B4QslXNLWRCLZUGHB_Dz9RjKGP8brkqFcpwdO97hb_0nqVFau7PVyBBGeQDzSmlFnKga98o6lutJODEz5d0ttl0knxtq0nF], either the token is invalid or SplunkD has exited : HTTP 401 -- <?xml version="1.0" encoding="UTF-8"?>
<response>
  <messages>
    <msg type="WARN">call not properly authenticated</msg>
  </messages>
</response>
INFO  Mapping - Loading mapping descriptors from jar:file:/opt/splunk/etc/apps/SPLUNK4JMX/bin/lib/jmxmodinput.jar!/mapping.XML
ERROR ModularInput - It has been determined via the REST API that all inputs have been disabled

Does someone have an idea why this problem occurs? I changed nothing on the heavy forwarder.
I can solve the Problem with a complete reinstallation of the heavy forwarder, but that cannot be the solution.

I am glad of any help.

Best regards,
Yanick

0 Karma
1 Solution

Damien_Dallimor
Ultra Champion

I can solve the Problem with a completreinstallation of the heavyforwarder.

Well , logic would tell me that it is something to do with your HF environment then.
Is your stanza name for your JMX input unique ? ie: there are no other inputs.conf stanzas anywhere in the deployment with the same name ?

View solution in original post

Damien_Dallimor
Ultra Champion

I can solve the Problem with a completreinstallation of the heavyforwarder.

Well , logic would tell me that it is something to do with your HF environment then.
Is your stanza name for your JMX input unique ? ie: there are no other inputs.conf stanzas anywhere in the deployment with the same name ?

kyaparla
Path Finder

Thanks for the inputs on this issue @Damien Dallimore

Could you please help with below scenario?

I am having similar issue running the app on splunk universal forwarder, running on s390x linux.

when I enable the app, it sends one set of jmx output and displays following error messages.

ERROR Can't connect to Splunk REST API with the token, either the token is invalid or SplunkD has exited : null

"ERROR It has been determined via the REST API that all inputs have been disabled"

and the PID from SPLUNK4JMX.pid file doesnt exist on running processes and doesnt trigger the modinput again ( I guess the inputs are disabled from the error message). Enabling app through command line triggers another one time jmx output.

0 Karma

simony
Path Finder

Yes, my stanza name for JMX Input is unique in the whole /opt/splunk Directory:

./etc/apps/SPLUNK4JMX/local/inputs.conf:[jmx://midw]

Meanwhile i could solve the problem with the following command:

/opt/splunk/bin/splunk enable app SPLUNK4JMX

Why this works, i d'ont know. In my default/app.conf the state is enabled:

[install]
state = enabled
is_configured = true

The enable command writes only an app.conf in the local directory with state = enabled.
Strange..

Regards,
Yanick

0 Karma

Damien_Dallimor
Ultra Champion

What does your jmx://midw stanza look like ?
Is the stanza enabled in local/inputs.conf ?

0 Karma

simony
Path Finder

Yes the stanza is enabled in the local/inputs.conf:

[jmx://midw]
config_file = config.xml
polling_frequency = 60
sourcetype = jmx
index = jmxitd
disabled = 0

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...