简体繁体 English

Kafka如何在经纪人之间分配主题分区

[英]How Kafka distributes the topic partitions among the brokers

原文 2015-09-04 07:56:40 0 1 apache-kafka/ kafka-consumer-api

I have 3 Kafka brokers in 3 different VMs, with one additionally running a Zookeeper. 我在3个不同的VM中有3个Kafka代理，其中一个还运行Zookeeper。 I now create a topic with 8 partitions. 我现在创建一个包含8个分区的主题。 The Producer pushes messages to these group of brokers on the created "topic". 生产者在创建的“主题”上将消息推送到这些代理组。

How does the Kafka distribute a topic and its partitions among the brokers? Kafka如何在经纪人之间分配主题及其分区？
Does the Kafka redistribute the topic when a new Kafka Broker joins a cluster? 当新的Kafka Broker加入群集时，Kafka是否会重新分发主题？
Can the topic partition be increased after the topic was created? 创建主题后是否可以增加主题分区？

1 个解决方案

When you create a new topic, Kafka places the partitions and replicas in a way that the brokers with least number of existing partitions are used first, and replicas for same partition are on different brokers. 当您创建新主题时，Kafka将分区和副本放置为首先使用具有最少数量的现有分区的代理，并且相同分区的副本位于不同的代理上。
When you add a new broker, it is used for new partitions (since it has lowest number of existing partitions), but there is no automatic balancing of existing partitions to the new broker. 添加新代理时，它将用于新分区（因为它具有最少数量的现有分区），但现有分区与新代理之间没有自动平衡。 You can use the replica-reassignment tool to move partitions and replicas to the new broker. 您可以使用副本重新分配工具将分区和副本移动到新代理。
Yes, you can add partitions to an existing topic. 是的，您可以将分区添加到现有主题。