简体   繁体   中英

Cannot get a joined Kafka stream to run or output anything

For the below code, both stream1 and stream2 run fine individually and I can see output, but the joined stream just doesn't log anything at all. I have a feeling it has something to do with the join window, but the data from both streams comes in at almost exactly the same time.

val stream = builder.stream(stringSerde, byteArraySerde, "topic")

val stream1 = stream
  .filter((key, value) => somefilter(key, value))
  .through(stringSerde, byteArraySerde, "topic1")

val stream2 = stream
  .filter((key, value) => someotherfilter(key, value))
  .through(stringSerde, byteArraySerde, "topic2")

val joinedStream = stream1
  .join(stream2, (value1: Array[Byte], value2: Array[Byte]) => {
    println("wont print anything")
    return somerandomdata
  },
  JoinWindows.of("othertopic").within(10000L),
  stringSerde, byteArraySerde, byteArraySerde)

Shouldn't the keys of both topic be te same in order to join them?

I think the Javadoc explains this : https://kafka.apache.org/0102/javadoc/org/apache/kafka/streams/kstream/JoinWindows.html

This might also be an interesting read : https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Streams+Join+Semantics

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM