如何在许多其他消费者使用的 apache kafka 集群中为我的消费者组选择 group.id？

Question

We have been provided with a kafka cluster url for consuming messages, but we as the a consumer-side have multiple cluster nodes.

For making sure that each message will be processed once only we are going to use a common group.id shared by our consumer group. Also we will Explicitly Commit Offsets.

My question is: how do we make sure that we will not pick a group.id that is already been used by other consumer groups? Is there a naming convention followed that ensures group.id uniqueness or should we contact the Kafka cluster maintainer for this?

Answer 1

There are character restrictions (like no spaces or symbols other than [_.-] group), but there is no general "convention" that is used outside of what your cluster maintainer might know.

You can view all existing groups using

kafka-consumer-groups --list

Answer 2

Many companies get around this problem by generally setting an application's Kafka consumer group.id to the application's name.