如果 ISR 小于复制因子并且生产者 acks 设置为 all，那么生产者会等待多少个 acks？

Question

• RF = 3 , ISR = 3 , acks = all >>> sent successfully
• RF = 3 , ISR = 2 , acks = all >>> sent successfully
• RF = 3 , ISR = 1 , acks = all >>> sent successfully
• RF = 3 , ISR = 1 , acks = all , min.isr.replicas = 3 >>> sent successfully !

So, If replication factor is 4, ISR is 3 and producer acks are set to all, How many acks will the producer wait for? I tried different scenarios, What should be the real behavior?

Answer 1

If you set acks=all broker which is the partition leader will wait all in-sync-replicas to replicate the data. In-sync-replica is a replica which is not far behind the partition leader.

What I mean by not far behind: In Kafka when a message is sent to a topic-partition (firstly message is received and stored in leader) and if replication factor for this topic is greater than 1, then replica broker(s) send fetch request(s) to leader broker and this data is replicated to other broker(s). If replica.lag.time.max.ms is passed from last caught up, replica is considered as out-of-sync and removed from ISR list. (It is still a replica and fetch messages but leader broker doesn't wait it until catch up and became an in-sync-replica again)

From Kafka docs:

Configuration parameter replica.lag.time.max.ms now refers not just to the time passed since last fetch request from replica, but also to time since the replica last caught up. Replicas that are still fetching messages from leaders but did not catch up to the latest messages in replica.lag.time.max.ms will be considered out of sync.

There is also min.insync.replicas parameter in broker config. It specifies minimum number of in-sync-replicas to continue sending message when acks=all .

min.insync.replicas: When a producer sets acks to "all" (or "-1"), min.insync.replicas specifies the minimum number of replicas that must acknowledge a write for the write to be considered successful. If this minimum cannot be met, then the producer will raise an exception (either NotEnoughReplicas or NotEnoughReplicasAfterAppend). When used together, min.insync.replicas and acks allow you to enforce greater durability guarantees. A typical scenario would be to create a topic with a replication factor of 3, set min.insync.replicas to 2, and produce with acks of "all". This will ensure that the producer raises an exception if a majority of replicas do not receive a write.

---

If replication factor is 4, ISR is 3 and producer acks are set to all, How many acks will the producer wait for?

Answer : Broker which is topic-partition leader will wait 3 other brokers in ISR list to replicate the data and send acknowledgement. If number of replicas in ISR list is less than min.insync.replicas then your producer get an exception and cannot produce the data.

Note: You can check current replica and ISR list with command below.

bin/kafka-topics.sh --bootstrap-server localhost:9092 --topic myTopic --describe

Answer 2

When it comes to the acks setting the replication factor of the topic itself only plays an implicit role. As written in the Broker Configuration documentation and cited below the min.insync.replicas defines the minimum number of successful replications broker-wide before it is seen as a successful write.

min.insync.replicas: When a producer sets acks to "all" (or "-1"), min.insync.replicas specifies the minimum number of replicas that must acknowledge a write for the write to be considered successful . If this minimum cannot be met, then the producer will raise an exception (either NotEnoughReplicas or NotEnoughReplicasAfterAppend). When used together, min.insync.replicas and acks allow you to enforce greater durability guarantees. A typical scenario would be to create a topic with a replication factor of 3, set min.insync.replicas to 2, and produce with acks of "all". This will ensure that the producer raises an exception if a majority of replicas do not receive a write.

Type: int

Default: 1

Valid Values: [1,...]

Importance: high

Update Mode: cluster-wide

To answer your conrete question: If replication factor is 4, and min.insync.replicas is 3 and producer acks are set to all, then the producer will wait for 3 acknowledgments within the cluster before it is seen as a successful write.