计算Kafka主题分区

Question

I have an n node Kafka cluster with just 2 topics I have replicated the topics across all n nodes I 'think' I have just a single consumer in the form of a mirrormaker consuming all topics although I intend to increase that from 1 to n mirror makers How many partitions should my topics use ? 1 then later n?

Answer 1

The number of nodes in your cluster (what you call "n") should be independent of the number of partitions in a topic (let's call that "p"). The max number of consumers in a consumer group (Mirror Maker or any other single group) will be p, but the number you actually need will be entirely driven by your throughput performance and message ordering requirements.