[英]How change topic leader or remove partition after some broker down?
We have kafka cluster with 4 brokers and some topics with replica factor 1 and 10 partitions.我们有 4 个代理的 kafka 集群和一些具有 1 个副本因子和 10 个分区的主题。 At one moment 2 of 4 our servers with kafka cluster - fail.
在某一时刻,我们的 4 台服务器中有 2 台带有 kafka 集群 - 失败。 So now we have 2 brokers with same topics.
所以现在我们有 2 个具有相同主题的经纪人。 When i
m run command./kafka_topics.sh --zookeeper localhost:2181 --describe i
m get this:当我
m run command./kafka_topics.sh --zookeeper localhost:2181 --describe i
得到这个:
Topic:outcoming-notification-error-topic PartitionCount:10 ReplicationFactor:1 Configs:
Topic: outcoming-error-topic Partition: 0 Leader: 2 Replicas: 2 Isr: 2
Topic: outcoming-error-topic Partition: 1 Leader: 3 Replicas: 3 Isr: 3
Topic: outcoming-error-topic Partition: 2 Leader: 4 Replicas: 4 Isr: 4
Topic: outcoming-error-topic Partition: 3 Leader: 1 Replicas: 1 Isr: 1
Topic: outcoming-error-topic Partition: 4 Leader: 2 Replicas: 2 Isr: 2
Topic: outcoming-error-topic Partition: 5 Leader: 3 Replicas: 3 Isr: 3
Topic: outcoming-error-topic Partition: 6 Leader: 4 Replicas: 4 Isr: 4
Topic: outcoming-error-topic Partition: 7 Leader: 1 Replicas: 1 Isr: 1
Topic: outcoming-error-topic Partition: 8 Leader: 2 Replicas: 2 Isr: 2
Topic: outcoming-error-topic Partition: 9 Leader: 3 Replicas: 3 Isr: 3
How can i delete Leader 2...4?如何删除领导者 2...4? or may be i need delete partition for this Leader, but how?
或者我可能需要删除这个领导者的分区,但是如何?
UPD..升级版..
Also we use kafka_exporter for monitoring kafka with prometheus.我们还使用 kafka_exporter 来监控带有 prometheus 的 kafka。 After 2 brokers down in log of kafka_exporter we get this errors:
在 kafka_exporter 的日志中记录了 2 个代理后,我们收到以下错误:
level=error msg="Cannot get oldest offset of topic outcoming-error-topic partition 10: kafka server: In the middle of a leadership election, there is currently no leader for this partition and hence it is unavailable for writes." source="kafka_exporter.go:296"
You could use Kafka's kafka-reassign-partitions.sh
to do that.你可以使用 Kafka 的
kafka-reassign-partitions.sh
来做到这一点。 You have two ways, one is generating a proposal of new assignments , and the other one is manually specifying the leaders for specific partitions.您有两种方法,一种是生成新分配的提案,另一种是手动指定特定分区的领导者。
The first method, as specified on the kafka docs , follows this logic:第一种方法,如kafka docs中所指定,遵循以下逻辑:
1.1 Generate Proposed partition reassignment configuration 1.1生成提议的分区重新分配配置
First, you should create a json file such as the provided in the link.首先,您应该创建一个 json 文件,如链接中提供的文件。 Let's name it
topics.json
.我们将其命名为
topics.json
。
{
"topics": [{"topic": "foo1"},
{"topic": "foo2"}],
"version":1
}
This will tell kafka what are the topics you are willing to rellocate their partitions from.这将告诉 kafka 您愿意从哪些主题中重新定位其分区。 In the example, he wants Kafka to make a proposal for topics
foo1
and foo2
.在示例中,他希望 Kafka 为主题
foo1
和foo2
提出建议。
With that json, call the tool and set the active broker list in the command:使用该 json,调用该工具并在命令中设置活动代理列表:
kafka-reassign-partitions.sh --zookeeper $ZK_HOSTS
--topics-to-move-json-file topics.json --broker-list "1,2,3,4,5" --generate
This will output Kafka's proposal, which you can save into another.json file.这将 output Kafka 的提案,您可以将其保存到另一个.json 文件中。 For example:
例如:
{
"version":1,
"partitions":[{"topic":"foo1","partition":2,"replicas":[5,6]},
{"topic":"foo1","partition":0,"replicas":[5,6]},
{"topic":"foo2","partition":2,"replicas":[5,6]},
{"topic":"foo2","partition":0,"replicas":[5,6]},
{"topic":"foo1","partition":1,"replicas":[5,6]},
{"topic":"foo2","partition":1,"replicas":[5,6]}]
}
You can manually modify some of the assignments, if you want to (or think it's the proper think to do, as the tool is not perfect).如果您愿意,您可以手动修改一些分配(或认为这是正确的想法,因为该工具并不完美)。 Save the json into a file, for example,
reassign-example.json
, which will be used in the next step.将 json 保存到文件中,例如
reassign-example.json
,将在下一步中使用。
1.2. 1.2. Execute the Proposed partition reassignment
执行提议的分区重新分配
Let's make Kafka execute the proposal and move the partitions.让我们让 Kafka 执行提案并移动分区。 For that, execute:
为此,执行:
bin/kafka-reassign-partitions.sh --zookeeper $ZK_HOSTS
--reassignment-json-file reassign-example.json --execute
This will execute the partition movement defined on the reassign-example.json
file.这将执行
reassign-example.json
文件中定义的分区移动。
The second method is fairly easier, but you must manually identify the partitions you want to reassign.第二种方法相当简单,但您必须手动识别要重新分配的分区。 For example, if you want partition 1 of topic XXX to move to brokers 5 and 6, you could create a json file (
manual-reassign.json
) such as:例如,如果您希望主题 XXX 的分区 1 移动到代理 5 和 6,您可以创建一个 json 文件(
manual-reassign.json
),例如:
{"version":1,"partitions":[{"topic":"XXX","partition":1,"replicas":[5,6]}]}
The way it's launched is the same as in the previous method:它的启动方式与之前的方法相同:
bin/kafka-reassign-partitions.sh --zookeeper $ZK_HOSTS
--reassignment-json-file manual-reassign.json --execute
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.