How to increase the number of partition for a Kafka Topic

Question

I'm using Kafka Spring to create some listeners for kafka topics. The topics are created automatically when app starts.

That's how a listener looks like:

  @KafkaListener(topics = "${kafka.topics.message.readFormatForMessageFromProfile.in}")

The problem is that i want to increase the number of partitions to 3 and i have no clue how to do that.

Answer 1

You can use topicPartitions attribute of @KafkaListner annotation.

@KafkaListener(id = "someId",
        topicPartitions =
            {
                @TopicPartition(topic = "${kafka.topics.message.readFormatForMessageFromProfile.in}",
                partitionOffsets = @PartitionOffset(partition = "0", initialOffset = "0"))})

You can add multiple @TopicPartition

Answer 2

I'm not an expert of Kafka Spring but I know how it's possible to do that using the official Admin Client API. By the way I see the following in the Kafka Spring documentation :

https://docs.spring.io/spring-kafka/reference/htmlsingle/#_configuring_topics

Reading this note :

If the broker supports it (1.0.0 or higher), the admin will increase the number of partitions if it is found that an existing topic has fewer partitions than the NewTopic.numPartitions.