简体   繁体   English

一个 Kafka 主题中的分区可以具有不同的复制因子吗?

[英]Can partitions in one Kafka topic have different replication factors?

When defininig a kafka topic, I can set the replication factor, and usually Kafka will distribute the partitions across the available brokers.在定义 kafka 主题时,我可以设置复制因子,通常 Kafka 会在可用的代理之间分配分区。

Say I have 3 brokers and have a topic with several partitions and a replication-factor=2, then some partitions will be set to be on brokers [0,1], some one [0,2] and some on [1,2] (ignoring the leader selection/preference for now)假设我有 3 个代理并且有一个包含多个分区和复制因子 = 2 的主题,那么一些分区将设置在代理 [0,1] 上,一些分区设置在 [0,2] 上,一些在 [1,2] 上](暂时忽略领导者选择/偏好)

Can I later eg set only one partition to have a replication factor of 3 (and thus add this partition to the remaining node) without having to also do this for the other partitions?我可以稍后仅将一个分区设置为具有 3 的复制因子(从而将此分区添加到剩余节点),而不必也为其他分区执行此操作? Is there any drawback to having a number !=all of the partitions replicated to more/less nodes than the others?有一个数字!=所有分区复制到比其他节点更多/更少的节点有什么缺点吗?

All documentation etc I saw only ever speaks of the replication factor in terms of whole topics我看到的所有文档等都只谈到了整个主题的复制因素

I know I use kafka-reassign-partitions on a per-partition basis, but I found no discussion or recommendations on whether it is good/bad to only reassign a few partitions or what the dangers (if any) might be我知道我在每个分区的基础上使用 kafka-reassign-partitions,但我没有发现关于只重新分配几个分区是好是坏或危险(如果有的话)可能是什么的讨论或建议

Only topics have replication factors, not partitions.只有主题有复制因子,没有分区。

Partitions can become out-of-sync , and lose leadership in the case of errors, in which case, one or more partitions becomes unavailable, but the topic is still the source of how many replicas there ought to be分区可能会变得不同步,并在出错的情况下失去领导力,在这种情况下,一个或多个分区变得不可用,但主题仍然是应该有多少副本的来源

I found no discussion or recommendations on whether it is good/bad to only reassign a few partitions我没有发现关于只重新分配几个分区是好是坏的讨论或建议

You'd usually only do this for maintenance or cluster expansion to remove large partitions from occupying too much disk space on certain brokers.您通常只会在维护或集群扩展时执行此操作,以删除大分区,以免占用某些代理上的过多磁盘空间。

The main "bad" thing would be how fast your network is to transfer the partition while still handling traffic to the other replicas of the topic.主要的“坏”事情是您的网络传输分区的速度有多快,同时仍然处理到主题的其他副本的流量。

声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM