简体繁体 English

5 节点 Broker 集群的 Kafka 复制因子

[英]Kafka Replication Factor For 5 node Broker Cluster

原文 2021-01-14 13:23:37 9 1 apache-kafka/ confluent-platform/ replication-factor

I am setting up a Confluent Kafka Cluster (Community) with 3 Zookeper and 5 Kafka Broker nodes.我正在设置一个具有 3 个 Zookeper 和 5 个 Kafka Broker 节点的 Confluent Kafka 集群（社区）。 The requirement is that we should be able to continue in live environment even if 2 broker nodes are down.要求是即使 2 个代理节点关闭，我们也应该能够在实时环境中继续。

What should be the recommended -replication factor, -in sync replica for topics with 50 partitions.对于具有 50 个分区的主题，推荐的复制因子、同步副本应该是多少。

In most case the suggested replication factor is 3. What would be the impact if we increase that to 5 in the mentioned cluster configuration在大多数情况下，建议的复制因子是 3。如果我们在提到的集群配置中将其增加到 5，会有什么影响

1 个解决方案

Setting the replication factor to 5 would mean that all partitions exist on all brokers in the cluster.将复制因子设置为 5 意味着所有分区都存在于集群中的所有代理上。 If two brokers are down, then the replication factor requirement is no longer met and your topics will be under-replicated (should give a warning).如果两个 broker 宕机，则不再满足复制因子要求，您的主题将被复制不足（应该给出警告）。

min.insync.replicas should then be set to 3 (or less), otherwise producing a message with acks = all would fail.然后应将min.insync.replicas设置为 3（或更少），否则生成带有acks = all的消息将失败。 Producing a message with acks set to 1, 2, or 3 would also work on higher values of min.insync.replicas .将acks设置为 1、2 或 3 来生成消息也适用于min.insync.replicas的更高值。

Also note that while two nodes are down, you can't create new topics with a replication factor of 5 (also see KIP-409 ).另请注意，当两个节点关闭时，您无法创建复制因子为 5 的新主题（另请参阅KIP-409 ）。