Kafka one consumer one partition

Question

I have a use case where I have a single topic with 100 partitions where messages go in each partition with some logic and I have 100 consumers who reads this message. I want to map a specific partition to a specific consumer. How can I achieve that?

Answer 1

Checkout the Javadoc for the KafkaConsumer, specifically the section "Manual Partition Assignment".

TL/DR

You can manually assign specific partitions to a consumer as follows:

String topic = "foo";
TopicPartition partition0 = new TopicPartition(topic, 0);
TopicPartition partition1 = new TopicPartition(topic, 1);
consumer.assign(Arrays.asList(partition0, partition1));