Kafka Streams：我可以将我的处理器订阅到具有不同分区数量的多个主题吗

Question

I'm trying to create a Kafka Stream Topology with multiple processors that subscribe to a regex pattern and multiple topics as a result.

For example, pattern topic_(?:a|b|c) will connect to topic_a, topic_b, and topic_c. I have a different number of partitions for each topic, since they have different loads. When running my topology, though, I see:

org.apache.kafka.streams.errors.TopologyException
...
Topics not co-partitioned

The difference between them are:

topic_a: 3 partitions
topic_b: 3 partitions
topic_c: 30 partitions

Am I allowed to do this, or when subscribing to multiple topics I need to maintain the same number of partitions?

Into further context, each of the topics listed above are actually two, like topic_a1 and topic_a2, topic_b1 and topic_b2. They all got the same partition number and I'm trying to join, and later aggregate them using the streams API

Answer 1

If an application is subscribed to multiple topics using a pattern, all of the topics need to have the same number of partitions.