Kafka consumers handle synchronous messages in different topic

Question

I have two Kafka producers create messages when creating and deleting customers. Each produces message in different topic. There exists some corresponding Kafka consumers that is responsible for actions create and delete customers.

The problem is when message delete customer C1 consumed before create C1. The customer C1 will be created and never get deleted.

How could I guarantee that delete message C1 is only consumed after create message C1 is consumed? Or other approach to assure that create and delete actions synchronized when we use kafka?

Answer 1

Ordering of data in Kafka is only guaranteed within a topic partition. If you ensure that all data (creation and deletion) for customer C1 goes to one partition (eg by choosing a kind of customer identifier as key of the Kafka message) you will ensure that the creation of data will be stored and therefore consumed before the deletion.

However, the problem that I see here is that you have two different producers. So there is always a chance that a deletion will be written into the Kafka topic before a creation (due to any kind of problems in the "creation" producer).

Ideally, you have one single producer covering both creations and deletions (if possible). I also suggest to combine the two consumers to the same ConsumerGroup.