Apache Kafka：如何找出主题的消费者群体？

Question

I am working on Kafka with Spark streaming in Scala. There are multiple topics from which I need to fetch messages. For that, I need to input consumer group of each topic.

I have tried following command:

bin/kafka-consumer-groups.sh --zookeeper localhost:2181 --describe --group group_name

For this, I need to mention consumer group and check whether my topic falls under this group. But I don't know the consumer group value for some topics.

So, is there any command or programmatic way of finding consumer group for a given topic?

Any help on this would be greatly appreciated. Thank you

Answer 1

You can read data from various topics by specifying the topics. You can simply create a Map of topics. Refer to the below link

https://www.tutorialspoint.com/apache_kafka/apache_kafka_consumer_group_example.htm

Answer 2

There is no Command/API provided by kafka to list consumer groups consuming from a specified topic. Instead you can use a tool developed by yahoo to manage kafka clusters : Yahoo Kafka Manager . After installing this software and pointing yahoo kafka manager to your kafka cluster, you can view consumer groups consuming from a specified topic.

Answer 3

for i in `kafka-consumer-groups --list --bootstrap-server localhost:9092`; do echo $i; (kafka-consumer-groups --describe --bootstrap-server localhost:9092 --group $i 2>&1| grep mytopic); done

这需要一些时间，因为它会检查所有主题，但会告诉哪些消费者群体使用该主题。