使用 kafka-consumer-groups.sh 脚本删除消费者组是删除消费者组元数据的安全方法吗？

Question

I am using multithreaded kafka consumers for a single 60 partitions kafka topic, with each consumer thread mapped to each partiton. After the consumption is complete, each consumer thread is closed using KafkaConsumer.close() method.

During reconsumption with same consumer groupid, (seek to beginning is used per consumer tread), 1 or 2 consumer threads fail to consume from their assigned partitions. If a new consumer group id is provided, this issue does not show up.

So, if the below script is used to delete a consumer group.
bin/kafka-consumer-groups.sh --bootstrap-servers "3 Servers:port" --delete --group group1

Is it a safe way to delete the consumer group metadata and if it also disconnects the active TCP connections from all consumers assigned to this group.

Please share your opinions on this.

Answer 1

If you always seek to the beginning of the topic, you could instead disable auto commit and set auto offset reset to earliest. Then your app always starts consuming from the beginning

Otherwise, I personally would reset the group rather than delete it

https://kafka.apache.org/documentation/#basic_ops_consumer_group

You can only reset and delete a group when there are no active consumers