[英]Confluent-Kafka Python : How to list all topics programmatically
I have a kafka broker running in my local system.我有一个在本地系统中运行的 kafka 代理。 To communicate with the broken using my Django based web application I am using confluent-kafka wrapper.
要使用我的基于 Django 的 web 应用程序与故障通信,我正在使用confluent-kafka包装器。 However by browsing through the admin api, I could not find out any api for listing kafka topics.
但是,通过浏览管理员 api,我找不到任何用于列出 kafka 主题的 api。 (The topics are created pragmatically and are dynamic).
(主题是务实地创建的并且是动态的)。
Is there any way I could list them within my program?有什么办法可以在我的程序中列出它们吗? The requirement is that, if my worker reboots all assigned consumers listening to those topics will have to be re-initialized so I want to loop to all the topics and assign a consumer to each.
要求是,如果我的工作人员重新启动所有指定的收听这些主题的消费者,则必须重新初始化,因此我想循环到所有主题并为每个主题分配一个消费者。
Two ways you can achieve it:您可以通过两种方式实现它:
from confluent_kafka import Consumer
consummer_config = {
'bootstrap.servers': 'localhost:9092',
'group.id': 'group1'
}
consumer = Consumer(consummer_config)
consumer.list_topics().topics
This will return a dictionary something like this:这将返回一个像这样的字典:
{'raw_request': TopicMetadata(raw_request, 4 partitions),
'raw_requests': TopicMetadata(raw_requests, 1 partitions),
'idw': TopicMetadata(idw, 1 partitions),
'__consumer_offsets': TopicMetadata(__consumer_offsets, 50 partitions)}
the other way is already posted by @NYCeyes另一种方式已经由@NYCeyes发布
Try searching again for list_topics
再次尝试搜索
list_topics
https://docs.confluent.io/5.2.1/clients/confluent-kafka-python/index.html#confluent_kafka.Consumer.list_topics https://docs.confluent.io/5.2.1/clients/confluent-kafka-python/index.html#confluent_kafka.Consumer.list_topics
https://docs.confluent.io/5.2.1/clients/confluent-kafka-python/index.html#confluent_kafka.Producer.list_topics https://docs.confluent.io/5.2.1/clients/confluent-kafka-python/index.html#confluent_kafka.Producer.list_topics
Here is how to do it:这是如何做到的:
>>> from confluent_kafka.admin import AdminClient
>>> conf = {'bootstrap.servers': 'vps01:9092,vps02:9092,vps03:9092'}
>>> kadmin = AdminClient(conf)
>>> kadmin.list_topics().topics # Returns a dict(). See example below.
{'topic01': TopicMetadata(topic01, 3 partitions),}
I hope that helps.我希望这有帮助。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.