JMS Modular Input - Saving an unaltered (working) input config results in disconnection?

[rturk]

Hi Splunkers! (well, probably one particular Splunker... hi Damian!)

You might remember me from such questions as 'JMS Modular Input 1.3.7 - Client ID already in use?', which I believe is very related.

To give some background, we have been able to create working a JMS input which is happily connecting to WebMethods environment.

When browsing the configuration of the working input via the GUI, if one clicks on the “SAVE” button (even without making any changes to the configuration), this seems to cause another connection attempt to the already established JMS queue (instead of checking if there is an existing connection and ignoring the save) this results in Splunk throwing the following error:

03-23-2015 09:25:48.453 +1100 ERROR ExecProcessor - message from "python "C:\Program Files\Splunk\etc\apps\jms_ta\bin\jms.py"" Stanza jms://topic/host:xxxJson_Receive : Error connecting : javax.jms.InvalidClientIDException: [BRM.10.2002] JMS: Client ID "DEV-SPLUNK" is already in use.

Which in turn results in not only failing to start the additional connection (which is expected/desired), but tears down the original functioning JMS input (possibly based on the duplicated Client ID?). However this tear down does not appear to be “clean”… (read on)

From what I can tell, WebMethods thinks the connection is still established. This is configured as a “DURABLE” (i.e. reliable) connection, so a disconnection should result in WebMethods queuing up messages until such a time as the connection is restored.

However, what IS happening is:

• WebMethods still sees Splunk as connected so messages are not queued and are “consumed” according to WebMethods and are not queued up when the connection drops.
• Splunk does not accept any messages (I have not done a packet capture to confirm they are even getting there, so this is an assumption on my behalf)
• Messages/events are lost with no avenue for recovery (boo)

It is also suspected (although yet to be confirmed) that this same action interrupts other existing inputs (the client has advised they are unable to have two simultaneous JMS inputs)

Is anyone else experiencing these type of issues? This may be a WebMethods thing, but would be interested to know if anyone else is having these issues.

Cheers & Beers!

RT

[Damien_Dallimor]

I'm not sure why you'd SAVE without any changes.

But anyway , this would cause the current modular input instance to end (hard process kill) and a new one to start up.
So any connections to your messaging server would end and new ones would be established.
I can not ignore a SAVE with "no changes" in the modular input code , this would be functionality required in core Splunk.

Regarding Client ID "DEV-SPLUNK" is already in use , this may well be vendor specific behaviour , Webmethods in this case. JMS is just an API that abstracts away many different underlying messaging platforms that have a JMS provider implementation.

This probably addresses the previous point , but regarding why Webmethods still sees an active connection/holds onto a dead connection even when the JMS Mod Input process terminates , hard to answer , may be Webmethods specific connection handling functionality ? Other messaging platforms seem to terminate the connection in this scenario.

I would suggest that if Webmethods removed the connection handle when the Mod Input terminates , this would probably address the core issue at hand regarding successful reconnection.

Regarding "...the client has advised they are unable to have two simultaneous JMS inputs...." , many usesr of this Mod Input have multiple inputs defined and running concurrently , so again , without knowing the exact deployment or seeing any config files , I can only suggest that this might be some constraint in the WebMethods setup ?