I am trying to connect to IBM Bluemix message-hub and produce a message using java following the example
https://github.com/ibm-messaging/message-hub-samples/tree/master/java/message-hub-kafka-ssl
producer.properties
key.serializer=org.apache.kafka.common.serialization.ByteArraySerializer
value.serializer=org.apache.kafka.common.serialization.ByteArraySerializer
acks=-1
security.protocol=SASL_SSL
sasl.mechanism=PLAIN
ssl.protocol=TLSv1.2
ssl.enabled.protocols=TLSv1.2
ssl.truststore.password=changeit
ssl.truststore.type=JKS
ssl.endpoint.identification.algorithm=HTTPS
ssl.truststore.location=/Library/Java/JavaVirtualMachines/jdk1.8.0_66.jdk/Contents/Home/jre/lib/security/cacerts
jaas.conf.template
KafkaClient {
com.ibm.messagehub.login.MessageHubLoginModule required
serviceName="kafka"
user="$USERNAME"
password="$PASSWORD";
};
Producer.java snippet
ProducerRecord<byte[], byte[]> record = new ProducerRecord<byte[], byte[]>(
"MyTopic",
KEY.getBytes("UTF-8"),
"MESSAGE".getBytes("UTF-8"));
// Synchronously wait for a response from Message Hub / Kafka.
RecordMetadata m = kafkaProducer.send(record).get();
The problem is , I am getting a timeout exception when I am trying to get the Future RecordMetadata
java.util.concurrent.ExecutionException: org.apache.kafka.common.errors.TimeoutException: Failed to update metadata after 60000 ms.
at org.apache.kafka.clients.producer.KafkaProducer$FutureFailure.<init>(KafkaProducer.java:730)
at org.apache.kafka.clients.producer.KafkaProducer.doSend(KafkaProducer.java:483)
at org.apache.kafka.clients.producer.KafkaProducer.send(KafkaProducer.java:430)
at org.apache.kafka.clients.producer.KafkaProducer.send(KafkaProducer.java:353)
Read an earlier post on the same topic
Timeout connecting to message-hub on Bluemix
And the possible reason mentioned was the topic was not created.I can see the topic in bluemix console and to verify, I called the rest service to fetch the list of topics before pushing the message
RESTRequest restApi = new RESTRequest(getRestHost(),getApiKey());
String topics = restApi.get("/admin/topics", false);
logger.info("Topics present in the system: " + topics);
It returns the topic where I am trying to push a message and yet I am getting the timeout error.
Can someone please help me in debugging the issue
UPDATE
Based on comments , I enabled debug logs for kafka, and here is the sequence of logs
2016-11-23 08:48:20.906 DEBUG 72885 --- [sage-hub-sample] org.apache.kafka.clients.NetworkClient : Initialize connection to node -5 for sending metadata request
2016-11-23 08:48:20.906 DEBUG 72885 --- [sage-hub-sample] org.apache.kafka.clients.NetworkClient : Initiating connection to node -5 at kafka05-prod01.messagehub.services.us-south.bluemix.net:9093.
2016-11-23 08:48:20.914 DEBUG 72885 --- [sage-hub-sample] o.a.k.c.s.a.SaslClientAuthenticator : Set SASL client state to SEND_HANDSHAKE_REQUEST
2016-11-23 08:48:20.915 DEBUG 72885 --- [sage-hub-sample] o.a.k.c.s.a.SaslClientAuthenticator : Creating SaslClient: client=messagehub/ignorehost@messagehub.ibm.com;service=kafka;serviceHostname=kafka05-prod01.messagehub.services.us-south.bluemix.net;mechs=[PLAIN]
2016-11-23 08:48:20.979 DEBUG 72885 --- [sage-hub-sample] org.apache.kafka.common.metrics.Metrics : Added sensor with name node--5.bytes-sent
2016-11-23 08:48:20.980 DEBUG 72885 --- [sage-hub-sample] org.apache.kafka.common.metrics.Metrics : Added sensor with name node--5.bytes-received
2016-11-23 08:48:20.980 DEBUG 72885 --- [sage-hub-sample] org.apache.kafka.common.metrics.Metrics : Added sensor with name node--5.latency
2016-11-23 08:48:20.982 DEBUG 72885 --- [sage-hub-sample] org.apache.kafka.clients.NetworkClient : Completed connection to node -5
2016-11-23 08:48:21.080 DEBUG 72885 --- [sage-hub-sample] o.a.k.c.s.a.SaslClientAuthenticator : Set SASL client state to RECEIVE_HANDSHAKE_RESPONSE
2016-11-23 08:48:21.264 DEBUG 72885 --- [sage-hub-sample] o.a.k.c.s.a.SaslClientAuthenticator : Set SASL client state to INITIAL
2016-11-23 08:48:21.265 DEBUG 72885 --- [sage-hub-sample] o.a.k.c.s.a.SaslClientAuthenticator : Set SASL client state to INTERMEDIATE
2016-11-23 08:48:21.284 DEBUG 72885 --- [sage-hub-sample] o.apache.kafka.common.network.Selector : Connection with kafka05-prod01.messagehub.services.us-south.bluemix.net/23.246.202.55 disconnected
java.io.EOFException: null
at org.apache.kafka.common.network.SslTransportLayer.read(SslTransportLayer.java:488)
at org.apache.kafka.common.network.NetworkReceive.readFromReadableChannel(NetworkReceive.java:81)
at org.apache.kafka.common.network.NetworkReceive.readFrom(NetworkReceive.java:71)
at org.apache.kafka.common.security.authenticator.SaslClientAuthenticator.receiveResponseOrToken(SaslClientAuthenticator.java:239)
at org.apache.kafka.common.security.authenticator.SaslClientAuthenticator.authenticate(SaslClientAuthenticator.java:182)
at org.apache.kafka.common.network.KafkaChannel.prepare(KafkaChannel.java:64)
at org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:318)
at org.apache.kafka.common.network.Selector.poll(Selector.java:283)
at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:260)
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:229)
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:134)
at java.lang.Thread.run(Thread.java:745)
All the above are logged for any of the 5 brokers. Also these are all debug statements so I am not sure whether they are errors.
-Tatha
I can see a couple of issues with your JAAS file:
The sample you're referring to is using Kafka 0.10.0.X so you should not use the old Message Hub login module for Kafka 0.9. So replace "com.ibm.messagehub.login.MessageHubLoginModule" by "org.apache.kafka.common.security.plain.PlainLoginModule"
The field for the user name is called "username" and not "user". It should be username="$USERNAME"
please enable log4j DEBUG level and the Kafka client trace may help identify any connectivity issues. Try change the last line to log4j.logger.org.apache.kafka=DEBUG
in the log4j.properties
file, rebuild and take a look at the verbose output. Feel free to repost that.
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.