如何在 JS 中为 Kafka 消费者获取分配的分区

Question

I'm working with a Kafka Consumer using KAFKAJS. I have a requirement to know all the partitions assigned to a specific consumer.

For example, let's say Topic 1 has 5 partitions. And there are 2 consumers with the same clientId. So, one will be assigned 3 topics and the other 2. I want each consumer to know the partitions assigned.

Answer 1

We can use consumer.describeGroup to query the state of the consumer group. https://kafka.js.org/docs/consuming#a-name-describe-group-a-describe-group

The memberAssignment field describes what topic-partitions are assigned to each member, but it's a Buffer, so you'll need to decode that using AssignerProtocol.MemberAssignment.decode:

https://github.com/tulios/kafkajs/blob/master/index.js#L18

We can also listen to the GROUP_JOIN event. It already contains the member assignment, so you don't need to actually describeGroup:

https://kafka.js.org/docs/instrumentation-events#consumer