简体   繁体   English

Kafka - 控制台消费者卡住了

[英]Kafka - Console consumer is stuck

I created simple setup with 3 brokers and then created a simple topic and pushed messages using console producer.我用 3 个代理创建了简单的设置,然后创建了一个简单的主题并使用控制台生产者推送消息。 I tried to consume messages using console consumer with all the brokers alive and running, consumer is stuck.我尝试使用控制台消费者消费消息,所有代理都处于活动状态并正在运行,消费者被卡住了。

Update #1更新 #1

And one of the broker is doing a lot of processing with exceptions such as "NotEnoughReplicasException"并且其中一位经纪人正在进行大量处理,但出现“NotEnoughReplicasException”等异常

Logs日志

    org.apache.kafka.common.errors.NotEnoughReplicasException: The size of the current ISR Set(2) is insufficient to satisfy the min.isr re
quirement of 2 for partition __consumer_offsets-10                                                                                     
[2020-06-19 17:36:24,249] INFO [GroupCoordinator 2]: Preparing to rebalance group console-consumer-28937 in state PreparingRebalance wi
th old generation 4822 (__consumer_offsets-10) (reason: error when storing group assignment during SyncGroup (member: consumer-console-
[2020-06-19 17:36:24,305] INFO [GroupCoordinator 2]: Stabilized group console-consumer-28937 generation 4823 (__consumer_offsets-10) (k
afka.coordinator.group.GroupCoordinator)                                                                                               
23 (kafka.coordinator.group.GroupCoordinator)                                                                                          
[2020-06-19 17:36:24,306] ERROR [ReplicaManager broker=2] Error processing append operation on partition __consumer_offsets-10 (kafka.s
erver.ReplicaManager)                                                                                                                  
org.apache.kafka.common.errors.NotEnoughReplicasException: The size of the current ISR Set(2) is insufficient to satisfy the min.isr re
quirement of 2 for partition __consumer_offsets-10                                                                                     
[2020-06-19 17:36:24,307] INFO [GroupCoordinator 2]: Preparing to rebalance group console-consumer-28937 in state PreparingRebalance wi
th old generation 4823 (__consumer_offsets-10) (reason: error when storing group assignment during SyncGroup (member: consumer-console-
consumer-28937-1-21df21a9-3e11-4286-8252-3871633cf3bd)) (kafka.coordinator.group.GroupCoordinator)                                     
[2020-06-19 17:36:24,349] INFO [GroupCoordinator 2]: Stabilized group console-consumer-28937 generation 4824 (__consumer_offsets-10) (k
afka.coordinator.group.GroupCoordinator)                                                                                               
[2020-06-19 17:36:24,351] INFO [GroupCoordinator 2]: Assignment received from leader for group console-consumer-28937 for generation 48
[2020-06-19 17:36:24,351] ERROR [ReplicaManager broker=2] Error processing append operation on partition __consumer_offsets-10 (kafka.s

Facts about the Topic关于主题的事实

Topic: new_topic        PartitionCount: 1       ReplicationFactor: 1    Configs:
    Topic: new_topic        Partition: 0    Leader: 3       Replicas: 3     Isr: 3

Fixed the issue by making min.insync.replicas = 1 ( the value was 2 while the issue is happening) as the topic in question new_topic had only replication factor 1 which is conflicting with min.insync.replicas .通过使min.insync.replicas = 1 (发生问题时值为2 )解决了该问题,因为有问题的主题new_topic只有复制因子 1 与min.insync.replicas冲突。

However, the point which I didnt get quite so far is how min.insync.replicas is interfering with consumer path.然而,到目前为止我还没有明白的一点是min.insync.replicas是如何干扰消费者路径的。

声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM