[英]Can I have all the consumers of a group consume message from all the partitions of a kafka topic?
Let's say in Kafka I have 4 partitions of a topic 'A' and I have 20 consumers of Consumer Group 'AC'. 假设在Kafka中,我有4个主题“ A”的分区,而我有20个“ AC”消费者组的消费者。 I don't need any ordering, but I want to process the messages faster by scaling my consumer instances. 我不需要任何排序,但是我想通过扩展使用者实例来更快地处理消息。 Please note all messages are independent and can be processed independently. 请注意,所有消息都是独立的,可以独立处理。
I looked at a consumer configuration partition.assignment.strategy
, but not sure if I can achieve dynamic assignment of consumer to partition, depending on the message availability. 我查看了使用者配置partition.assignment.strategy
,但不确定是否可以根据消息可用性将使用者动态分配到分区。
One partition is assigned to exactly one consumer in the group. 一个分区分配给组中的一个使用者。 In your case you have only 4 consumers on 20 which are currently working. 在您的情况下,您只有20个工作中的4个消费者。 You have to increase partitions number if you want more assigned consumers. 如果要分配更多的使用者,则必须增加分区数。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.