修改Kafka主题复制因子后会发生什么？

Question

According to the docs , you just modify a topic to increase the replication factor.

 > bin/kafka-topics.sh --zookeeper zk_host:port/chroot --create --topic my_topic_name 
       --partitions 20 --replication-factor 3 --config x=y

Unfortunately, it doesn't specify what happens then after you modify the topic. Do existing log segments get replicated to the new replicas, or only new messages?

Answer 1

For kafka 10 you need to invoke the kafka-reassign-partitions.sh script to change the replication factor of the topic. Here is a demo of a script that will display the topic before and after the change:

updateTopicReplication() {
    TOPIC_NAME=$1
    REPLICAS=$2

    echo "****************************************"
    echo "describe $TOPIC_NAME"
    bin/kafka-topics.sh --zookeeper localhost:2181 --describe --topic $TOPIC_NAME

    JSON_FILE=./configChanges/${TOPIC_NAME}-update.json
    echo $JSON_FILE
    [ -e $JSON_FILE ] && rm $JSON_FILE

    touch $JSON_FILE

    echo -e "{\"version\":1, \"partitions\":[{\"topic\":\"${TOPIC_NAME}\",\"partition\":0,\"replicas\":[${REPLICAS}]}]}" >> $JSON_FILE

    echo "****************************************"
    echo "updateing $TOPIC_NAME"
    bin/kafka-reassign-partitions.sh --zookeeper localhost:2181 --reassignment-json-file $JSON_FILE --execute

    echo "****************************************"
    echo "describe $TOPIC_NAME"
    bin/kafka-topics.sh --zookeeper localhost:2181 --describe --topic $TOPIC_NAME
}

Answer 2

根据我在问题中提到的同一份文档，似乎kafka实际上不支持增加（或减少）主题的复制因子。