如何防止 kafka 到 mqtt Apache Camel 路由失敗？

Question

我有一個將 Kafka 流數據分發到 MQTT 主題的系統。 Apache Camel 路由如下所示：

<?xml version="1.0" encoding="UTF-8"?>
<routes xmlns="http://camel.apache.org/schema/spring">
   <route id="KafkaToMQTT">
      <from uri="kafka://mqtt?brokers=localhost:9092" />
      <to uri="micrometer:timer:camel.proxy.kafka.mqtt.stream?action=start" />
      <toD uri="mqtt:mqtt?host=tcp://localhost:1883&amp;publishTopicName=${header.kafka.KEY}" />
      <to uri="log://camel.proxy?groupInterval=100&amp;level=INFO" />
      <to uri="micrometer:timer:camel.proxy.kafka.mqtt.stream?action=stop" />
   </route>
</routes>

並且由於數據流開始，對於 10 個主題的 7 個主題，一切正常，但對於應該屬於 3 個其他主題的消息（不同的運行之間不同，但在同一個駱駝路線運行中相同）我收到錯誤消息使用如下堆棧跟蹤：

19:19:34.195 [Camel (camel-1) thread #4 - KafkaConsumer[mqtt]] ERROR o.a.c.processor.DefaultErrorHandler - Failed delivery for (MessageId: ID-camel-proxy-586888f9b6-dfnwp-1652296571794-0-14406 on ExchangeId: ID-camel-proxy-586888f9b6-dfnwp-1652296571794-0-14404). Exhausted after delivery attempt: 1 caught: java.lang.IllegalStateException: Already connected

Message History
---------------------------------------------------------------------------------------------------------------------------------------
RouteId              ProcessorId          Processor                                                                        Elapsed (ms)
[KafkaToMQTT       ] [KafkaToMQTT       ] [kafka://mqtt?brokers=kafka%3A9092&groupId=mqtt                                ] [         0]
[KafkaToMQTT       ] [to85              ] [micrometer:timer:camel.proxy.kafka.mqtt.stream?action=start                   ] [         0]
[KafkaToMQTT       ] [toD1              ] [                                                                              ] [         0]

Stacktrace
---------------------------------------------------------------------------------------------------------------------------------------
java.lang.IllegalStateException: Already connected
        at org.fusesource.mqtt.client.CallbackConnection.connect(CallbackConnection.java:135)
        at org.apache.camel.component.mqtt.MQTTEndpoint.connect(MQTTEndpoint.java:315)
        at org.apache.camel.component.mqtt.MQTTProducer.ensureConnected(MQTTProducer.java:108)
        at org.apache.camel.component.mqtt.MQTTProducer.process(MQTTProducer.java:39)
        at org.apache.camel.processor.SendDynamicProcessor$1.doInAsyncProducer(SendDynamicProcessor.java:178)
        at org.apache.camel.impl.ProducerCache.doInAsyncProducer(ProducerCache.java:445)
        at org.apache.camel.processor.SendDynamicProcessor.process(SendDynamicProcessor.java:160)
        at org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:548)
        at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:201)
        at org.apache.camel.processor.Pipeline.process(Pipeline.java:138)
        at org.apache.camel.processor.Pipeline.process(Pipeline.java:101)
        at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:201)
        at org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:97)
        at org.apache.camel.component.kafka.KafkaConsumer$KafkaFetchRecords.doRun(KafkaConsumer.java:326)
        at org.apache.camel.component.kafka.KafkaConsumer$KafkaFetchRecords.run(KafkaConsumer.java:215)
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:829)

對我來說，不清楚什么樣的交互會中斷，如果只是連接池不足等資源瓶頸或其他什么，可以配置什么來增加資源？

Answer 1

由於動態性質，這看起來像是<toD/>相關問題。 使用預定義的值檢查進行顯式切換使一切正常：

<routes
    xmlns="http://camel.apache.org/schema/spring">
    <route id="KafkaToMQTT">
        <from uri="kafka://mqtt?brokers=kafka:9092"/>
        <to uri="micrometer:timer:camel.proxy.kafka.mqtt.stream?action=start"/>
                <when>
                    <simple>${header.kafka.KEY} == "ExpectedValue1"</simple>
                    <to uri="mqtt:mqtt?host=tcp://mqtt:1883&amp;publishTopicName=ExpectedValue1"/>
                </when> 
                <when>
                    <simple>${header.kafka.KEY} == "ExpectedValue2"</simple>
                    <to uri="mqtt:mqtt?host=tcp://mqtt:1883&amp;publishTopicName=ExpectedValue2"/>
                </when> 
        <to uri="log://camel.proxy?groupInterval=100&amp;level=INFO"/>
        <to uri="micrometer:timer:camel.proxy.kafka.mqtt.stream?action=stop"/>      
</route>
</routes>

這不是那么普遍，但對我的目標有用。