Partitioned Kafka Topic for DLQ for spring-cloud-stream

Question

When a DLQ is set up for a spring-cloud-stream Kafka consumer, can the topic the DLQ writes to be partitioned? I have a requirement to make the key equal to a specific field and I was wondering how that would be possible with spring-cloud-stream.

Answer 1

The topic provisioner in the Kafka binder will create the DLQ topic with the same number of partitions configured for the corresponding consumer topic. When messages in error are sent to the DLQ, it then appropriately send them to the same partition where the consumer received record from the original topic.

If you are creating the DLQ outside of the provisioner, you might want to make sure that you create the DLQ with the same number of partitions as the consumer topic.