KSQL - Join unequal partitions streams

Question

How to join unequal number of partition holding streams in KSQL apart from increase the partition ?

Example Stream-1 is having the 3 partitions and Stream-2 is having the 2 partitions . In that case , of course we can increase the number partitions for Stream-1 as 3 join . But I want to know , any other method to join unequal partitioned streams through KSQL ?

Answer 1

No, unfortunately KStream/KSQL doesn't support join for unequal partitioned topics.

It's a pre-requisite that both topics should have same number of partitions before calling join operation otherwise it will fail.

You can read more about Co-partitioning requirement here: https://docs.confluent.io/current/ksql/docs/developer-guide/partition-data.html#partition-data-to-enable-joins

To ensure co-partitioning, you can use PARTITION_BY clause to create new stream :

CREATE STREAM topic_rekeyed WITH (PARTITIONS=6) AS SELECT * FROM topic PARTITION BY topic_key;