Given that, there are number of posts in this theme, I am probably doing Splunk right. 🙂
Here is the log from splunkd.log
02-06-2015 16:43:23.737 +0530 INFO SpecFiles - Found external scheme definition for stanza "mqtt://" with 15 parameters: topic_name, broker_host, broker_port, use_ssl, username, passwor
d, client_id, qos, reliable_delivery_dir, clean_session, connection_timeout, keepalive_interval, message_handler_impl, message_handler_params, additional_jvm_propertys
02-06-2015 16:43:23.737 +0530 INFO SpecFiles - Found external scheme definition for stanza "perfmon://" with 11 parameters: object, counters, instances, interval, mode, samplingInterval
, stats, disabled, index, showZeroValue, useEnglishOnly
02-06-2015 16:43:24.649 +0530 INFO ModularInputs - Endpoint argument settings for "additional_jvm_propertys":
02-06-2015 16:43:24.649 +0530 INFO ModularInputs - Endpoint argument settings for "broker_host":
02-06-2015 16:43:24.649 +0530 INFO ModularInputs - Endpoint argument settings for "broker_port":
02-06-2015 16:43:24.649 +0530 INFO ModularInputs - Endpoint argument settings for "clean_session":
02-06-2015 16:43:24.649 +0530 INFO ModularInputs - Endpoint argument settings for "client_id":
02-06-2015 16:43:24.649 +0530 INFO ModularInputs - Endpoint argument settings for "connection_timeout":
02-06-2015 16:43:24.649 +0530 INFO ModularInputs - Endpoint argument settings for "keepalive_interval":
02-06-2015 16:43:24.649 +0530 INFO ModularInputs - Endpoint argument settings for "message_handler_impl":
02-06-2015 16:43:24.649 +0530 INFO ModularInputs - Endpoint argument settings for "message_handler_params":
02-06-2015 16:43:24.649 +0530 INFO ModularInputs - Endpoint argument settings for "name":
02-06-2015 16:43:24.649 +0530 INFO ModularInputs - Endpoint argument settings for "password":
02-06-2015 16:43:24.649 +0530 INFO ModularInputs - Endpoint argument settings for "qos":
02-06-2015 16:43:24.649 +0530 INFO ModularInputs - Endpoint argument settings for "reliable_delivery_dir":
02-06-2015 16:43:24.649 +0530 INFO ModularInputs - Endpoint argument settings for "topic_name":
02-06-2015 16:43:24.649 +0530 INFO ModularInputs - Endpoint argument settings for "use_ssl":
02-06-2015 16:43:24.649 +0530 INFO ModularInputs - Endpoint argument settings for "username":
02-06-2015 16:43:24.649 +0530 INFO ModularInputs - Introspection setup completed for scheme "mqtt".
followed, later, by:
02-06-2015 16:43:32.296 +0530 ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/mqtt_ta/bin/mqtt.py" Can't connect to Splunk REST API with the token [Splunk UCsOBihUByd91WO
^mn3L8noZwWJDbpkwcY7Hvf2EWv2W^C0YLBPU95xsleT0Kp4UhWkxRo3Xw_urAqala45dhR8L8gaGad7FoFr8OKNux3mLH_xFCB2WPRN], either the token is invalid or SplunkD has exited : No appropriate protocol (pr
otocol is disabled or cipher suites are inappropriate)
02-06-2015 16:43:42.298 +0530 ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/mqtt_ta/bin/mqtt.py" It has been determined via the REST API that all inputs have been disab
led
Going by a similar post around JMX, I checked the JAVA_HOME variable - looks ok.
champ@champ-vm:~$ java -version
java version "1.8.0_31"
Java(TM) SE Runtime Environment (build 1.8.0_31-b13)
Java HotSpot(TM) 64-Bit Server VM (build 25.31-b07, mixed mode)
champ@champ-vm:~$ echo $JAVA_HOME
/usr/lib/jvm/java-8-oracle
All said and done, I am pretty sure, this is not a Java problem and in fact, the MQTT connection is probably going on correctly. I tried cloning the MQTT input definition using the Clone link. However, I forgot to change the client identifier. Now, as per the MQTT spec, the client identifier should be unique across a broker. If not, then the formerly connected client will be disconnected. The Eclipse Paho client does provide a call-back upon a disconnect, where, I guess, a re-connect attempt is made. But, since the client ID is same again, now, the recently connected client is forcibly disconnected - trigerring a call back and so on.
I am saying this, because, I did see some exception messages as shown below:
02-06-2015 16:59:46.874 +0530 ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/mqtt_ta/bin/mqtt.py" Stanza mqtt://localbroker : Error disconnecting : Client is disconnecte
d (32101)
02-06-2015 16:59:46.874 +0530 ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/mqtt_ta/bin/mqtt.py" at org.eclipse.paho.client.mqttv3.internal.ExceptionHelper.createM
qttException(ExceptionHelper.java:27)
02-06-2015 16:59:46.874 +0530 ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/mqtt_ta/bin/mqtt.py" at org.eclipse.paho.client.mqttv3.internal.ClientComms.disconnect(
ClientComms.java:405)
02-06-2015 16:59:46.874 +0530 ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/mqtt_ta/bin/mqtt.py" at org.eclipse.paho.client.mqttv3.MqttAsyncClient.disconnect(MqttA
syncClient.java:524)
02-06-2015 16:59:46.874 +0530 ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/mqtt_ta/bin/mqtt.py" at org.eclipse.paho.client.mqttv3.MqttClient.disconnect(MqttClient
.java:250)
02-06-2015 16:59:46.874 +0530 ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/mqtt_ta/bin/mqtt.py" at org.eclipse.paho.client.mqttv3.MqttClient.disconnect(MqttClient
.java:243)
02-06-2015 16:59:46.874 +0530 ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/mqtt_ta/bin/mqtt.py" at com.splunk.modinput.mqtt.MQTTModularInput$MessageReceiver.disco
nnect(Unknown Source)
02-06-2015 16:59:46.874 +0530 ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/mqtt_ta/bin/mqtt.py" at com.splunk.modinput.mqtt.MQTTModularInput$MessageReceiver.run(U
nknown Source)
02-06-2015 16:59:46.936 +0530 ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/mqtt_ta/bin/mqtt.py" Stanza mqtt://Local : Error running message receiver : Connection lost
(32109) - java.io.EOFException
02-06-2015 16:59:46.936 +0530 ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/mqtt_ta/bin/mqtt.py" at org.eclipse.paho.client.mqttv3.internal.CommsReceiver.run(Comms
Receiver.java:138)
02-06-2015 16:59:46.936 +0530 ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/mqtt_ta/bin/mqtt.py" at java.lang.Thread.run(Thread.java:745)
02-06-2015 16:59:46.936 +0530 ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/mqtt_ta/bin/mqtt.py" Caused by: java.io.EOFException
02-06-2015 16:59:46.936 +0530 ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/mqtt_ta/bin/mqtt.py" at java.io.DataInputStream.readByte(DataInputStream.java:267)
02-06-2015 16:59:46.936 +0530 ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/mqtt_ta/bin/mqtt.py" at org.eclipse.paho.client.mqttv3.internal.wire.MqttInputStream.re
adMqttWireMessage(MqttInputStream.java:56)
02-06-2015 16:59:46.936 +0530 ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/mqtt_ta/bin/mqtt.py" at org.eclipse.paho.client.mqttv3.internal.CommsReceiver.run(Comms
Receiver.java:100)
02-06-2015 16:59:46.936 +0530 ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/mqtt_ta/bin/mqtt.py" ... 1 more
The Troubleshooting on the MQTT Modular Input page is not very helpful.
Run this command as the same user that
you are running Splunk as and observe
console output :
"$SPLUNK_HOME/bin/splunk cmd python
../etc/apps/mqtt_ta/bin/mqtt.py
--scheme"
Ok, ran the command and observed - very pretty! Now, what?
<scheme>
<title>MQTT</title>
<description>Index messages from a MQTT Broker</description>
<use_external_validation>true</use_external_validation>
<streaming_mode>xml</streaming_mode>
<use_single_instance>true</use_single_instance>
<endpoint>
<args>
<arg name="name">
<title>Stanza Name</title>
<description></description>
<data_type>string</data_type>
<required_on_edit>false</required_on_edit>
<required_on_create>true</required_on_create>
</arg>
<arg name="topic_name">
<title>Topic Name</title>
<description></description>
<data_type>string</data_type>
<required_on_edit>false</required_on_edit>
<required_on_create>true</required_on_create>
</arg>
<arg name="broker_host">
<title>Broker Host</title>
<description></description>
<data_type>string</data_type>
<required_on_edit>false</required_on_edit>
<required_on_create>true</required_on_create>
</arg>
<arg name="broker_port">
<title>Broker Port</title>
<description></description>
<data_type>string</data_type>
<required_on_edit>false</required_on_edit>
<required_on_create>false</required_on_create>
</arg>
<arg name="use_ssl">
<title>Use SSL</title>
<description></description>
<data_type>string</data_type>
<required_on_edit>false</required_on_edit>
<required_on_create>false</required_on_create>
</arg>
<arg name="username">
<title>Username</title>
<description></description>
<data_type>string</data_type>
<required_on_edit>false</required_on_edit>
<required_on_create>false</required_on_create>
</arg>
<arg name="password">
<title>Password</title>
<description></description>
<data_type>string</data_type>
<required_on_edit>false</required_on_edit>
<required_on_create>false</required_on_create>
</arg>
<arg name="client_id">
<title>Client ID</title>
<description></description>
<data_type>string</data_type>
<required_on_edit>false</required_on_edit>
<required_on_create>false</required_on_create>
</arg>
<arg name="qos">
<title>QOS</title>
<description></description>
<data_type>string</data_type>
<required_on_edit>false</required_on_edit>
<required_on_create>false</required_on_create>
</arg>
<arg name="reliable_delivery_dir">
<title>Reliable Delivery Directory</title>
<description></description>
<data_type>string</data_type>
<required_on_edit>false</required_on_edit>
<required_on_create>false</required_on_create>
</arg>
<arg name="clean_session">
<title>Clean Session</title>
<description></description>
<data_type>string</data_type>
<required_on_edit>false</required_on_edit>
<required_on_create>false</required_on_create>
</arg>
<arg name="connection_timeout">
<title>Connection Timeout</title>
<description></description>
<data_type>string</data_type>
<required_on_edit>false</required_on_edit>
<required_on_create>false</required_on_create>
</arg>
<arg name="keepalive_interval">
<title>Keep Alive Interval</title>
<description></description>
<data_type>string</data_type>
<required_on_edit>false</required_on_edit>
<required_on_create>false</required_on_create>
</arg>
<arg name="additional_jvm_propertys">
<title>Additional JVM Propertys</title>
<description></description>
<data_type>string</data_type>
<required_on_edit>false</required_on_edit>
<required_on_create>false</required_on_create>
</arg>
<arg name="message_handler_impl">
<title>Implementation class for a custom message handler</title>
<description>An implementation of the com.splunk.modinput.mqtt.AbstractMessageHandler class.You would provide this if you required some custom handling/formatting of the messages you consume.Ensure that the necessary jars are in the $SPLUNK_HOME/etc/apps/mqtt_ta/bin/lib directory</description>
<data_type>string</data_type>
<required_on_edit>false</required_on_edit>
<required_on_create>false</required_on_create>
</arg>
<arg name="message_handler_params">
<title>Implementation parameter string for the custom message handler</title>
<description>Parameter string in format 'key1=value1,key2=value2,key3=value3'. This gets passed to the implementation class to process.</description>
<data_type>string</data_type>
<required_on_edit>false</required_on_edit>
<required_on_create>false</required_on_create>
</arg>
</args>
</endpoint>
</scheme>
Can you please help me resolve this ?
... View more