如何解決 Spring Cloud Stream Kafka Streams Binder 中多路復用輸入主題的 InvalidTopicException？

Question

我寫了一個 Spring Cloud Streams Kafka Streams Binder 應用程序，它有多個 Kafka 輸入主題多路復用到一個 stream ：

spring:
  cloud:
    stream:
      bindings:
        process-in-0:
          destination: test.topic-a,test.topic-b

（來源： https://spring.io/blog/2019/12/03/stream-processing-with-spring-cloud-stream-and-apache-kafka-streams-part-2-programming-model-continued ）

但是每當我在輸入目標中設置多個主題（以逗號分隔）時，就會出現以下錯誤：

2022-06-17 14:07:07.648  INFO --- [-StreamThread-1] o.a.k.clients.consumer.KafkaConsumer     : [Consumer clientId=test-processor-2ba8d1d3-5bbe-45d3-a832-6a24cf2f5549-StreamThread-1-consumer, groupId=test-processor] Subscribed to topic(s): test-processor-KTABLE-AGGREGATE-STATE-STORE-0000000005-repartition, test.topic-a,test.topic-b  
2022-06-17 14:07:07.660  WARN --- [-StreamThread-1] org.apache.kafka.clients.NetworkClient   : [Consumer clientId=test-processor-2ba8d1d3-5bbe-45d3-a832-6a24cf2f5549-StreamThread-1-consumer, groupId=test-processor] Error while fetching metadata with correlation id 2 : {test-processor-KTABLE-AGGREGATE-STATE-STORE-0000000005-repartition=UNKNOWN_TOPIC_OR_PARTITION, test.topic-a,test.topic-b=INVALID_TOPIC_EXCEPTION}  
2022-06-17 14:07:07.660 ERROR --- [-StreamThread-1] org.apache.kafka.clients.Metadata        : [Consumer clientId=test-processor-2ba8d1d3-5bbe-45d3-a832-6a24cf2f5549-StreamThread-1-consumer, groupId=test-processor] Metadata response reported invalid topics [test.topic-a,test.topic-b]  
2022-06-17 14:07:07.660  INFO --- [-StreamThread-1] org.apache.kafka.clients.Metadata        : [Consumer clientId=test-processor-2ba8d1d3-5bbe-45d3-a832-6a24cf2f5549-StreamThread-1-consumer, groupId=test-processor] Cluster ID: XYZ 
2022-06-17 14:07:07.663 ERROR --- [-StreamThread-1] org.apache.kafka.streams.KafkaStreams    : stream-client [test-processor-2ba8d1d3-5bbe-45d3-a832-6a24cf2f5549] Encountered the following exception during processing and Kafka Streams opted to SHUTDOWN_CLIENT. The streams client is going to shut down now.   

org.apache.kafka.streams.errors.StreamsException: org.apache.kafka.common.errors.InvalidTopicException: Invalid topics: [test.topic-a,test.topic-b]
    at org.apache.kafka.streams.processor.internals.StreamThread.runLoop(StreamThread.java:627) ~[kafka-streams-3.2.0.jar:na]
    at org.apache.kafka.streams.processor.internals.StreamThread.run(StreamThread.java:551) ~[kafka-streams-3.2.0.jar:na]
Caused by: org.apache.kafka.common.errors.InvalidTopicException: Invalid topics: [test.topic-a,test.topic-b]

我嘗試了以下依賴項：

implementation 'org.apache.kafka:kafka-clients:3.2.0'
implementation 'org.apache.kafka:kafka-streams:3.2.0'
implementation "org.springframework.cloud:spring-cloud-stream"
implementation "org.springframework.cloud:spring-cloud-stream-binder-kafka"
implementation "org.springframework.cloud:spring-cloud-stream-binder-kafka-streams"
implementation "org.springframework.kafka:spring-kafka"

當我只設置一個輸入主題時，一切正常。

我無法確定導致 InvalidTopicException 的原因，因為我只在主題名稱中使用允許的字符，而且逗號分隔符似乎是正確的（否則會出現不同的異常）。

Answer 1

實際上，在發布問題后，我自己找到了/一個解決方案/解決方法。 所以這是為了將來的幫助：

顯然，當我的處理器拓撲需要KTable作為輸入類型時，我不允許多路復用輸入主題。 當我將處理器簽名更改為KStream時，它突然起作用了：

不工作：

@Bean
public Function<KTable<String, Object>, KStream<String, Object>> process() {
  return stringObjectKTable ->
    stringObjectKTable
      .mapValues(...

在職的：

@Bean
public Function<KStream<String, Object>, KStream<String, Object>> process() {
  return stringObjectKStream ->
    stringObjectKStream 
      .toTable()
      .mapValues(...

我不確定，如果這是預期的行為，或者是否有其他問題，所以如果有更多的潛在問題，我很感激任何提示。