kafka-python错误获取请求密钥

Question

I am having trouble connecting to Cloudera Kafka version 2.0.1 (0.9.9.0) by using kafka-python versions both 1.2.4. and 1.2.5. My python version is 2.6.6. Below is the command and exception stack trace:

>>> from kafka import KafkaProducer

>>> producer = KafkaProducer(bootstrap_servers='host')


2016-07-20 10:56:29,370 ERROR kafka.network.Processor: Closing socket for ###:9092 - ###:50257 because of error
kafka.network.InvalidRequestException: Error getting request for apiKey: 18 and apiVersion: 0
    at kafka.network.RequestChannel$Request.liftedTree2$1(RequestChannel.scala:93)
    at kafka.network.RequestChannel$Request.<init>(RequestChannel.scala:90)
    at kafka.network.Processor$$anonfun$run$11.apply(SocketServer.scala:426)
    at kafka.network.Processor$$anonfun$run$11.apply(SocketServer.scala:421)
    at scala.collection.Iterator$class.foreach(Iterator.scala:742)
    at scala.collection.AbstractIterator.foreach(Iterator.scala:1194)
    at scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
    at scala.collection.AbstractIterable.foreach(Iterable.scala:54)
    at kafka.network.Processor.run(SocketServer.scala:421)
    at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: Unexpected ApiKeys id `18`, it should be between `0` and `16` (inclusive)
    at org.apache.kafka.common.protocol.ApiKeys.forId(ApiKeys.java:69)
    at org.apache.kafka.common.requests.AbstractRequest.getRequest(AbstractRequest.java:39)
    at kafka.network.RequestChannel$Request.liftedTree2$1(RequestChannel.scala:90)
    ... 9 more

I have verified I can connect by using kafka-console utilities on this machine, and also using the same versions of kafka-python on another machine. Has anyone seen this error or know what the problem may be?

Thank you.

Answer 1

What makes you suspect a connection error in kafka-python? Was there some client error, or maybe other strange behavior?

The error you pasted is from the broker, and simply reflects an attempt to identify the broker version by probing various newer api calls. You can safely ignore the error. To prevent the version probe, pass an explicit api_version to your KafkaProducer. I assume api_version=(0, 9) is what you want here, but I'm not actually familiar with the vendored kafka distribution you have (Cloudera Kafka version 2.0.1).