confluent-kafka python avro消息

Question

So, I can successfully use the python confluent-kafka api to consume messages from Kafka that are text encoded.

When I try to consume Avro encoded (binary) messages, I get a UnicodeDecodeError exception on:

msg = kafka_consumer.poll(timeout=2.0)

Obviously, Avro encoded values are binary, not Unicode. With the Java API, I can specify a KafkaAvroDeserializer to the KafkaConsumer constructor. The Python API seems to have no parallel or similar configuration option to this.

The configuration options that can be passed to the underlying librdkafka don't seem to specify any custom (de)serializer or decoder or avro related options.

https://github.com/edenhill/librdkafka/blob/master/CONFIGURATION.md

How do I consume Avro encoded messages with the Python confluent-kafka api?

Answer 1

在irc上，＃kafka的一位开发人员表示，这是confluent-python 3.0发行版中的一个错误，已在master分支中进行了修复，而该修复将在下一个发行版中进行。