[英]kafka-python raise UnrecognizedBrokerVersion Error
I am getting this error when constructing KafkaProducer with the kafka-python
package:使用kafka-python
package 构建 KafkaProducer 时出现此错误:
[ERROR] UnrecognizedBrokerVersion: UnrecognizedBrokerVersion
Traceback (most recent call last):
File "/var/lang/lib/python3.7/imp.py", line 234, in load_module
return load_source(name, filename, file)
File "/var/lang/lib/python3.7/imp.py", line 171, in load_source
module = _load(spec)
File "<frozen importlib._bootstrap>", line 696, in _load
File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 728, in exec_module
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "/var/task/kafka/producer/kafka.py", line 381, in __init__
**self.config)
File "/var/task/kafka/client_async.py", line 240, in __init__
self.config['api_version'] = self.check_version(timeout=check_timeout)
File "/var/task/kafka/client_async.py", line 908, in check_version
version = conn.check_version(timeout=remaining, strict=strict, topics=list(self.config['bootstrap_topics_filter']))
File "/var/task/kafka/conn.py", line 1228, in check_version
raise Errors.UnrecognizedBrokerVersion()
The code is as follows:代码如下:
from kafka import KafkaProducer
producer = KafkaProducer(bootstrap_servers=os.environ.get('KAFKA_HOST', 'localhost:9092'))
I am using Python 3.7 and an AWS MSK cluster.我正在使用 Python 3.7 和 AWS MSK 集群。
Solved it by just adding security_protocol="SSL"
to the KafkaProducer
as follows:通过将security_protocol="SSL"
添加到KafkaProducer
来解决它,如下所示:
from kafka import KafkaProducer
producer = KafkaProducer(security_protocol="SSL", bootstrap_servers=os.environ.get('KAFKA_HOST', 'localhost:9092'))
There are lots of optional parameters with the kafka-python
lib methods. kafka-python
lib 方法有很多可选参数。 Here's a recent working script with which I was able to listen to an Azure server:这是一个最近的工作脚本,我可以用它来收听 Azure 服务器:
import os, kafka # pip install kafka-python
consumer = kafka.KafkaConsumer("test-topic",
bootstrap_servers=["test-server.servicebus.windows.net:9093"],
auto_offset_reset="earliest",
enable_auto_commit=True,
group_id="$Default",
sasl_mechanism="PLAIN",
sasl_plain_password=os.environ["KAFKA_PASSWORD"],
sasl_plain_username="$ConnectionString",
security_protocol="SASL_SSL",
value_deserializer=lambda x: x.decode("utf-8"))
print(datetime.datetime.now())
for message in consumer:
message_dict = json.loads(message.value)
print(datetime.datetime.now())
print("%s" % (json.dumps(message_dict,indent=4)))
It is crude, but effective.它很粗糙,但很有效。 Simpler than the Java-wrapped-in-bash Kafka demo consumer from the https://kafka.apache.org/quickstart page (which required assorted config files and environment variables.)比https://kafka.apache.org/quickstart页面中的 Java-wrapped-in-bash Kafka 演示使用者更简单(需要各种配置文件和环境变量。)
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.