Spring-Kafka usage of ConcurrentKafkaListenerContainerFactory for more than One @Kafkalistener

Question

I am working on implementing consumption of messages from Kafka Topics using Spring-Kafka framework. I am trying to understand some usage of the ConcurrentKafkaListenerContainerFactory that i am creating for my Kafka Listener. The @KafkaListener works fine and as expected, however, in my scenario, i have more than one independent Listeners, listening to more than one Topic respectively. I would like to know if i can reuse the ConcurrentKafkaListenerContainerFactory among all my Listeners, or do i have to create one containerFactory per @KafkaListener. Is there a way of having a generic containerFactory that can be shared among all @Kafkalisteners

Thanks you

Answer 1

Yes; that's the whole point - it's a factory for listener containers; you typically only need the one factory that boot auto configures.

If you need different properties (eg deserializers) for a listener, recent versions (since spring-kafka 2.2.4) allow you to override consumer properties on the annotation.

To override other properties, eg container properties, for individual listeners, add a listener container customizer to the factory.

@Component
class ContainerFactoryCustomizer {

    ContainerFactoryCustomizer(AbstractKafkaListenerContainerFactory<?, ?, ?> factory) {
        factory.setContainerCustomizer(
                container -> {
                    String groupId = container.getContainerProperties().getGroupId();
                    if (groupId.equals("foo")) {
                        container.getContainerProperties().set...
                    }
                    else {
                        container.getContainerProperties().set...
                    }
                });
    }

As you can see, you can tell which container we are creating when it is called by accessing the groupId() container property.

You might want to use 2 factories if your listeners have vastly different configuration, but then you lose boot's auto configuration features (at least for the factory).