卡夫卡分区和生产者关系

Question

I have a kafka cluster with three brokers and one topic with replication factor of three and three partitions. I can see that every broker has a copy of log for all partitions with the same size. There are two producers for this topic.

One day I reduced writing volume of one producer by half. Then I found that all three brokers' inbound traffic reduced which is expected, but only partition 1's leader node's out traffic reduced which I don't understand.

The partition leader's outbound traffic reduced because of replication. But each broker is the leader of one partition, why only one leader's outbound traffic reduced? Is it possible that the producer only writes content to one partition? while I don't think so.

Please help me explain it. The cluster is working fine now, but I need to understand it in case of potential problem.

Answer 1

Assuming you are using Default Partitioner for KafkaProducer, which means two events with the same key are guaranteed to be sent to the same partition.

From From Kafka Documentation

All reads and writes go to the leader of the partition and Followers consume messages from the leader just as a normal Kafka consumer would and apply them to their own log.

You could have reduced data ( from a producer) by skiping specific key or set of Keys, which could means no data to particular partition.

This answers why leader's outbound traffic reduced (No records for followers to consume)