简体   繁体   中英

Find the topics under a consumer group in Kafka using Kafka-Python

Input to my python script is the name of a consumer group and the output is supposed to be the list of topics under the consumer group.
My code currently, doesn't return the topics for which the current offset is -1.
Is there a better way, by which I can obtain the list of all topics under the consumer group using Kafka-python.
I am actually looking to replace describe a topic in a group cmd using Kafka tools.

from kafka import KafkaAdminClient
topics_in_groups = {};
client = KafkaAdminClient(bootstrap_server='localhost:9092')
for group in client.list_consumer_groups():
  topics_in_groups[group[0]] = [];
  topic_dict = client.list_consumer_group_offsets(group[0]);
  for topic in topic_dict:
    topics_in_groups[group[0]].append(topic.topic)

client.list_consumer_group_offsets(group[0]) will get you the metadata of consumer group, the below code will give you the list of topics and partitions of a group.

topics_in_groups = []
for topic,prtitions in topic_dict:
    topics_in_groups.append({"topic":topic, "partition": partition})

I tried this and worked.

from kafka import KafkaAdminClient
topics_in_groups = {}
client = KafkaAdminClient(bootstrap_servers=['broker:9092'])

for group in client.list_consumer_groups():
    topics_in_groups[group[0]] = []

for group in topics_in_groups.keys():
    my_topics = []
    topic_dict = client.list_consumer_group_offsets(group)
    for topic in topic_dict:
        my_topics.append(topic.topic)
        topics_in_groups[group] = list(set(my_topics))

for key , value in topics_in_groups.items():
    print(key, "\n\t", value)

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM