如果我在 kafka 中有复制因子为 3 的单个代理怎么办？

Question

If I am creating a new cluster with 1 broker 1 topic 1 partition and replication factor as 3,then what happens ?It creates 3 replicas(Partition) under that single broker ? If yes,how the leader is elected ?

Answer 1

If you try to create a topic with a replication factor larger than the available number of brokers in your clusters you will get an Exception as below.

> ./kip-kafka-topics --create --topic multiple_replicas_on_single_broker --replication-factor 3 --partitions 10

Error while executing topic command : org.apache.kafka.common.errors.InvalidReplicationFactorException: Replication factor: 3 larger than available brokers: 1.
[2020-09-15 17:45:42,649] ERROR java.util.concurrent.ExecutionException: org.apache.kafka.common.errors.InvalidReplicationFactorException: Replication factor: 3 larger than available brokers: 1.
        at org.apache.kafka.common.internals.KafkaFutureImpl.wrapAndThrow(KafkaFutureImpl.java:45)
        at org.apache.kafka.common.internals.KafkaFutureImpl.access$000(KafkaFutureImpl.java:32)
        at org.apache.kafka.common.internals.KafkaFutureImpl$SingleWaiter.await(KafkaFutureImpl.java:89)
        at org.apache.kafka.common.internals.KafkaFutureImpl.get(KafkaFutureImpl.java:260)
        at kafka.admin.TopicCommand$AdminClientTopicService.createTopic(TopicCommand.scala:175)
        at kafka.admin.TopicCommand$TopicService.createTopic(TopicCommand.scala:134)
        at kafka.admin.TopicCommand$TopicService.createTopic$(TopicCommand.scala:129)
        at kafka.admin.TopicCommand$AdminClientTopicService.createTopic(TopicCommand.scala:157)
        at kafka.admin.TopicCommand$.main(TopicCommand.scala:60)
        at kafka.admin.TopicCommand.main(TopicCommand.scala)
Caused by: org.apache.kafka.common.errors.InvalidReplicationFactorException: Replication factor: 3 larger than available brokers: 1.

In the Kafka documentaion in the section Replication it is explained that the replication is done for server failures:

Kafka replicates the log for each topic's partitions across a configurable number of servers (you can set this replication factor on a topic-by-topic basis). This allows automatic failover to these replicas when a server in the cluster fails so messages remain available in the presence of failures.

Therefore, if you place more than one replca on a single server it would not be able to recover if that one server goes down.

---

Looking at your question "It creates 3 replicas(Partition) under that single broker ?" I would also recommend to clearly understand the difference between a replica and a partition . This post might help you.

Answer 2

You are welcome to try yourself...

You will get an error saying replicas is more than available brokers