自动更改kafka主题分区领导者

Question

I have an issue with my Kafka cluster. I have 3 brokers, so when I stop the broker 1 (for example), each topic partition with leader 1 change his leader with the seconde broker in replica configuration. So this is the good behavior and it works fine.

But when I restart the broker 1 I need to execute:

./kafka-preferred-replica-election.sh --zookeeper myHost

because the current leader is the other replica.

So my question is : there is a way to configure Kafka to do it automatically ?

thx

Answer 1

I'm assuming your default (when all brokers are running) assignment is balanced, and the preferred leaders are evenly spread.

Yes Kafka can re-elect the preferred leaders for all the partitions automatically when a broker is restarted. This is actually enabled by default, see auto.leader.rebalance.enable .

Upon restarting a broker, Kafka can take up to leader.imbalance.check.interval.seconds to trigger the re-election. This defaults to 5 minutes. So maybe you just did not wait long enough!

There is also leader.imbalance.per.broker.percentage which defines the percentage of non-preferred leaders allowed. This default to 10%.

For the full details about these configuration, see the broker config section on Kafka's website.