kafka leader unavailable - kafka-reassign-partitions

Question

Following some tutorials / courses about the Apache Kafka I have a question about reassigning the partitions. CLI kafka-reassign-partitions reassigns the leaders and partitions, looks good.

However it was mentions that this command can be used to in case a partition leader broker is not available to elect another partition leader.

Question: If a partition leader broker (say any broker) is not available, shouldn't a new leader be elected automatically? Isn't it responsibility of the controller node? Or was it just a wrongly intended usage?

Answer 1

You are right, Kafka should take care itself to select a new partition leader. Any in-sync replica can be used as a new partition leader. However, it might happen that the partition leader broker died before it was able to replicate all data to its replicas. In that case the CLI you mentioned can help.

However, this tool has two main usages (note that Kafka expects a failed broker to be re-started and it will not automatically shift all data from a healthy replication to a running broker):

• imagine you have a cluster with five brokers and you apply a replication factor of three. Over time you are creating multiple topics and those topics grow in byte size each individually. When creating the topic, Kafka does not know how large the topic will get and it selects the leader on a round-robin basis. To avoid having all the "big" topics on one broker and run out of disk space you can reassing its partitions (including leader and replicas) to other brokers using that tool.

• If you add more brokers to your cluster, Kafka will still stick to the existing brokers for their partition leaders (and replicas). Only newly created topics can make use of the new brokers. Even when you restart all brokers, the partition leaders will stick to their old location. The CLI tool helps you to balance out the load accross all (including the new) brokers.