简体   繁体   English

使用 Kafka-Python 在 Kafka 中查找消费者组下的主题

[英]Find the topics under a consumer group in Kafka using Kafka-Python

Input to my python script is the name of a consumer group and the output is supposed to be the list of topics under the consumer group.我的 python 脚本的输入是消费者组的名称,output 应该是消费者组下的主题列表。
My code currently, doesn't return the topics for which the current offset is -1.我的代码目前不返回当前偏移量为 -1 的主题。
Is there a better way, by which I can obtain the list of all topics under the consumer group using Kafka-python.有没有更好的方法,我可以通过Kafka-python获取消费者组下所有主题的列表。
I am actually looking to replace describe a topic in a group cmd using Kafka tools.我实际上正在寻找使用 Kafka 工具替换描述组 cmd 中的主题。

from kafka import KafkaAdminClient
topics_in_groups = {};
client = KafkaAdminClient(bootstrap_server='localhost:9092')
for group in client.list_consumer_groups():
  topics_in_groups[group[0]] = [];
  topic_dict = client.list_consumer_group_offsets(group[0]);
  for topic in topic_dict:
    topics_in_groups[group[0]].append(topic.topic)

client.list_consumer_group_offsets(group[0]) will get you the metadata of consumer group, the below code will give you the list of topics and partitions of a group. client.list_consumer_group_offsets(group[0])将为您获取消费者组的元数据,以下代码将为您提供组的主题和分区列表。

topics_in_groups = []
for topic,prtitions in topic_dict:
    topics_in_groups.append({"topic":topic, "partition": partition})

I tried this and worked.我试过这个并且工作。

from kafka import KafkaAdminClient
topics_in_groups = {}
client = KafkaAdminClient(bootstrap_servers=['broker:9092'])

for group in client.list_consumer_groups():
    topics_in_groups[group[0]] = []

for group in topics_in_groups.keys():
    my_topics = []
    topic_dict = client.list_consumer_group_offsets(group)
    for topic in topic_dict:
        my_topics.append(topic.topic)
        topics_in_groups[group] = list(set(my_topics))

for key , value in topics_in_groups.items():
    print(key, "\n\t", value)

声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM