Exactly once Producer and Consumption - Apache Kafka and SpringBoot

Question

I am working on a microservices project. In this project, there is a Microservice A is doing a process in various steps. At the completion of each step, Microservice sends a message into a kafka topic. Then another Microservice B consumes the message from the kafka topic and sends an email notifying the successful completion of the step. I need Exactly once semantics for this. I am using KafkaTemplate.send in Microservice A and @KafkaListener to read the message in Microservice B. My question is whether KafkaTemplate producer and @KafkaListener consumer are idempotent and if not, how can I make them idempotent.

Regards,

I am creating autowiring the KafkaTemplate using the following code:- @Autowired public EventProducer(NewTopic topic, KafkaTemplate<String, Event> kafkaTemplate) { this.kafkaTemplate = kafkaTemplate; }

Answer 1

Exactly once semantics in Kafka apply to consume->process->produce operations within the same application - even then, only the entire cpp is "exactly once"; the consume->process part is at least once; consumption is always at least once (or at most once), including in your scenario (for B).