[英]Kafka Streams join by key with complex condition
I'm trying to join KStream
with GlobalKTable
by key, but with specific logic.我正在尝试通过键将KStream
与GlobalKTable
连接KStream
,但具有特定的逻辑。
StreamsBuilder builder = new StreamsBuilder();
KStream<String, Integer> stream = builder.stream(inputTopic1); // key = "ABC"
GlobalKTable<String, Integer> table = builder.globalTable(inputTopic2); // key = "ABC"
stream.join(table, // join first by "ABC" = "ABC", then by "AB" = "AB", then by "A" = "A"
(key, value) -> key,
(valueLeft, valueRigth) -> {/* identify by which condition the join was performed */});
For example, if the key = "ABC", then:例如,如果 key = "ABC",则:
Additionally, it is required to know by which condition was the join performed - eg, by 3 letters / by 2 letters / by 1 letter.此外,还需要知道执行连接的条件 - 例如,通过 3 个字母 / 通过 2 个字母 / 通过 1 个字母。
The question is, is it possible at all or should I search for a workaround?问题是,是否有可能,或者我应该寻找解决方法? For example, make copies of GlobalKTable with corresponding keys (table with "ABC" key, one with "AB" key and one with "A" key) and perform 3 separate joins?例如,使用相应的键(带有“ABC”键的表,带有“AB”键的表和带有“A”键的表)制作 GlobalKTable 的副本并执行 3 个单独的连接? Or maybe any other suggestions?或者也许还有其他建议?
Thanks in advance!提前致谢!
Using a series of left-joins against multiple tables would be possible (if you know of often you want to try the join).对多个表使用一系列左连接是可能的(如果您知道经常想尝试连接)。 If the joins succeeds, you skip the next join.如果连接成功,则跳过下一个连接。 Using a combination of leftJoin()
and branch()
should allow you split the stream after each join into "joined" and "retry".使用leftJoin()
和branch()
应该允许您在每次加入后将流拆分为“加入”和“重试”。 At the end, you can merge()
the different result streams together if you want.最后,如果需要,您可以merge()
不同的结果流合并在一起。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.