Spring和Kafka：加入3个Kafka主题以生成输出Kafka流

Question

I have a requirement to join 3 Kafka topics. First two topic A and B will be added using inner join as the message key is same and generate a new Kafka stream with POJO same as B. Now with this accumulated stream, I need to join another topic C and I need to group the output based on a field , which is present in C.

So far I have below approach for this :

KStream - KStream inner join for first two topic ( A and B) Will it be possible to not publish this accumulated output on any topic and still be able to use it below

KStream - KStream (Above accumulated stream and topic C)

Could you please suggest a better approach or any examples that I can look on similar implementation in java.

Answer 1

You can use two consecutive joins:

KStream streamAB = streamA.join(streamB, ...);
// either:
KStream streamABC = streamA.selectKey(...) // set to the key as in streamC
                           .join(streamC, ...);
// or:
KStream streamCNew = streamC.selectKey(...); // set to the key as in streamAB
KStream streamABC = streamA.join(streamCnew, ...);
// or:
KStream streamCNew = streamC.selectKey(...); // set to a new join key
KStream streamABC = streamA.selectKey(...) // set to a new join key
                           .join(streamC, ...);

streamABC.selectKey(/* extract grouping field and set as key */).to("outputTopic");