Passing Acknowledgment to a spring KafkaListener consumer method

Question

I am trying to turn off auto committing in kafka, and instead do it manually. To that end, in my application.properties I have set spring.kafka.properties.enable.auto.commit=false

I also currently have a method with the following header:

@KafkaListener(id="${"+ KafkaConfiguration.APP_REQUEST_ID +"}", topics = "${"+KafkaConfiguration.PPA_REQUEST_TOPIC +"}")
public void receive(@Payload String message,
                    @Headers MessageHeaders headers)

My understanding is that in order to manually commit I need access to the Acknowledgement object, which would be passed in as a parameter to my receive() method. My question: if I change the header to

@KafkaListener(id="${"+ KafkaConfiguration.APP_REQUEST_ID +"}", topics = "${"+KafkaConfiguration.APP_REQUEST_TOPIC +"}")
public void receive(@Payload String message,
                    @Headers MessageHeaders headers,
                    Acknowledgment acknowledgment)

Will the Acknowledgment automatically be passed in, or are there other changes I need to make?

Answer 1

yes, that way an Acknowledgment instance would be passed into your listener method. after successful processing of the received message you should call acknowledgement.acknowledge(); (only needed if you want to manually ack)

I'd also switch to MANUAL ackmode and turn off auto-commit (what you already did), eg by providing a custom Spring Boot configuration class - maybe also configurable via application.properties :

@Configuration
class KafkaConfiguration {

        @Bean
        ConcurrentKafkaListenerContainerFactory<String, String> kafkaListenerContainerFactory(KafkaProperties kafkaProperties) {

            final Map<String, Object> consumerProperties = kafkaProperties.buildConsumerProperties();
            consumerProperties.put(ENABLE_AUTO_COMMIT_CONFIG, false);

            ConcurrentKafkaListenerContainerFactory<String, String> factory = new ConcurrentKafkaListenerContainerFactory<>();
            factory.getContainerProperties().setAckMode(MANUAL);

            return factory;
        }
    }
}

If you do NOT want to manually acknowledge then a different ackmode might be more convenient and a better fit:

https://docs.spring.io/spring-kafka/api/org/springframework/kafka/listener/ContainerProperties.AckMode.html

AckMode.RECORD is quite comfortable, since the Kafka record that was passed into your listener method will be automatically be acked if your listener's method implementation completes successfully (no exception is thrown).

Answer 2

Short answer, you do not need to pass Acknowledgment , it is part of the received message.

As per the documentation

When using manual AckMode , you can also provide the listener with the Acknowledgment . The following example also shows how to use a different container factory.

@KafkaListener(id = "cat", topics = "myTopic",
          containerFactory = "kafkaManualAckListenerContainerFactory")
public void listen(String data, Acknowledgment ack) {
    ...
    ack.acknowledge();
}

From @KafkaListener documentation:

Annotated methods are allowed to have flexible signatures similar to what MessageMapping provides, that is

• ConsumerRecord to access to the raw Kafka message
• Acknowledgment to manually ack
• @Payload-annotated method arguments including the support of validation
• @Header-annotated method arguments to extract a specific header value, defined by KafkaHeaders
• @Headers-annotated argument that must also be assignable to Map for getting access to all headers.
• MessageHeaders arguments for getting access to all headers.
• MessageHeaderAccessor for convenient access to all method arguments.

Answer 3

What if we want to pass in a spring managed bean via method injection? Im trying to pass in a service class via method injection to a kafka listener -

private fun defaultListener(payload: ByteArray, @Headers messageHeaders: MessageHeaders, ack: Acknowledgment, callbackService: CallbackService) {
 // Do something
}

I get the following exception -

org.springframework.messaging.converter.MessageConversionException: Cannot handle message; nested exception is org.springframework.messaging.converter.MessageConversionException: Cannot convert from [[B] to [com.test.kafka-demo.service.CallbackService]

Works fine if i make the dependent service class Autowired.