Kafka multiple producers to specific same partition

Question

Using spring kafka template, I have 2 different producers that send different messages with the same key to a topic always in the same form:

Producer 1: Sends Key: 1 message: abc partition 0

Producer 2: Sends Key: 2 message: def

I have 3 partitions and all messages are sent to the same according to the message key.

Now, I need to make sure, that according to some properties, specific messages will go to specific partitions to be able to manage some priority in the system.

Thing is Producer 2 doesn't have a way to know which partition the Producer 1 selected.

Kafka assures that messages from different producers sent to with the same key will go to the same partition. But when assigning the partition from the producer 1, producer 2 sends the message with the same key to a different partition.

I've seen information about this, but none with the producer selecting the partition that will use.

Shouldn't kafka keep assigning the same partition to the message even though producer ones picks the partition it will use?

Answer 1

This post's answer might help you understand the issue you are facing. offset of partition 0 is very close to be the sum of offset from rest partitions In substance

/** * The default partitioning strategy: * *

If a partition is specified in the record, use it *

If no partition is specified but a key is present choose a partition based on a hash of the key *

If no partition or key is present choose a partition in a round-robin fashion */