The Zookeeper servers are Linux instances which are hosted outside Aviva Network, On the other had the sPlunk Search heads are installed on AWS Cloud instances in the Aviva domain.
We established a successful connectivity to the Non-Prod zookeeper servers and were able to index data to Splunk. However when we tried connecting to the Production Zookeeper instance , we started getting the below error. We have confirmed that there is data in the KAFKA Prod servers.
The KAFKA team has confirmed that the sPlunk server has established a connection to the zookeeper however we are not able to read any data from there. We also tried doing a telnet from the Splunk server to Zookeeper host and port and it works successfully.
Error Message Seen :
2/26/18
2:25:57.113 PM
02-26-2018 14:25:57.113 +0000 ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/kafka_ta/bin/kafka.py" SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
host = 10-64-133-150.avivaaws.com source = /opt/splunk/var/log/splunk/splunkd.log sourcetype = splunkd
2/26/18
2:25:57.113 PM
02-26-2018 14:25:57.113 +0000 ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/kafka_ta/bin/kafka.py" SLF4J: Defaulting to no-operation (NOP) logger implementation
host = 10-64-133-150.avivaaws.com source = /opt/splunk/var/log/splunk/splunkd.log sourcetype = splunkd
2/26/18
2:25:57.112 PM
02-26-2018 14:25:57.112 +0000 ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/kafka_ta/bin/kafka.py" SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
host = 10-64-133-150.avivaaws.com source = /opt/splunk/var/log/splunk/splunkd.log sourcetype = splunkd
2/26/18
2:25:50.669 PM
02-26-2018 14:25:50.669 +0000 INFO ExecProcessor - New scheduled exec process: python /opt/splunk/etc/apps/kafka_ta/bin/kafka.py
host = 10-64-133-150.avivaaws.com source = /opt/splunk/var/log/splunk/splunkd.log sourcetype = splunkd
2/26/18
2:25:50.664 PM
02-26-2018 14:25:50.664 +0000 INFO IndexWriter - openDatabases complete currentId=1 idx=
connectivity details:-
[root@prodms02 kafka]# netstat -anp|grep 2181|grep '10.64.'
tcp 0 0 10.48.178.195:2181 10.64.139.58:46312 ESTABLISHED 7448/java
kafka server splunk server
Could you please suggest the solution as soon as possible.
