Spring 云 Stream - 如何处理下游块？

Question

During a planned downtime for our Kafka cluster, we basically encountered the following issue How to specify timeout for sending message to RabbitMQ using Spring Cloud Stream? (with Kafka rather than RabbitMQ, obviously).

The answer from @GaryRussell:

The channel sendTimeout only applies if the channel itself can block, eg a QueueChannel with a bounded queue that is currently full; the caller will block until either space becomes available in the queue, or the timeout occurs.

In this case, the block is downstream of the channel so the sendTimeout is irrelevant (in any case, it's a DirectChannel which can't block anyway, the subscribed handler is called directly on the calling thread).

The actual blocking you are seeing is most likely in the socket.write() in the rabbitmq client, which does not have a timeout and is not interruptible; there is nothing that can be done by the calling thread to "time out" the write.

The only possible solution I am aware of is to force close the rabbit connection by calling resetConnection() on the connection factory.

explains quite well why the method in question ( org.springframework.integration.channel.AbstractSubscribableChannel#doSend ) does not take the timeout into account. However, this still seems a bit odd to me.

In spring-integration-kafka-3.2.1.RELEASE-sources.jar./org/springframework/integration/kafka/outbound/KafkaProducerMessageHandler:java:566 , we can see that, if sync behaviour is desired:

565    if (this.sync) {
566        Long sendTimeout = this.sendTimeoutExpression.getValue(this.evaluationContext, message, Long.class);
567        if (sendTimeout == null || sendTimeout < 0) {
568            future.get();
569        }
570        else {
571            try {
572                future.get(sendTimeout, TimeUnit.MILLISECONDS);
573            }
574            catch (TimeoutException te) {
575                throw new MessageTimeoutException(message, "Timeout waiting for response from KafkaProducer", te);
576            }
577        }
578    }

is called, where a timeout is taken into consideration. The sendTimeoutExpression is assigned to a default Value:

    private static final long DEFAULT_SEND_TIMEOUT = 10000;
    private Expression sendTimeoutExpression = new ValueExpression<>(DEFAULT_SEND_TIMEOUT);

however, our stack trace reveals something different:

"pool-1-thread-3" - Thread t@108
   java.lang.Thread.State: TIMED_WAITING
    at sun.misc.Unsafe.park(Native Method)
    - parking to wait for <4ebda621> (a org.springframework.util.concurrent.SettableListenableFuture$SettableTask)
    at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
    at java.util.concurrent.FutureTask.awaitDone(FutureTask.java:426)
    at java.util.concurrent.FutureTask.get(FutureTask.java:204)
    at org.springframework.util.concurrent.SettableListenableFuture.get(SettableListenableFuture.java:134)
*   at org.springframework.integration.kafka.outbound.KafkaProducerMessageHandler.processSendResult(KafkaProducerMessageHandler.java:572)
    at org.springframework.integration.kafka.outbound.KafkaProducerMessageHandler.handleRequestMessage(KafkaProducerMessageHandler.java:414)
    at org.springframework.integration.handler.AbstractReplyProducingMessageHandler.handleMessageInternal(AbstractReplyProducingMessageHandler.java:134)
    at org.springframework.integration.handler.AbstractMessageHandler.handleMessage(AbstractMessageHandler.java:69)
    at org.springframework.cloud.stream.binder.AbstractMessageChannelBinder$SendingHandler.handleMessageInternal(AbstractMessageChannelBinder.java:1035)
    at org.springframework.integration.handler.AbstractMessageHandler.handleMessage(AbstractMessageHandler.java:69)
    at org.springframework.integration.dispatcher.AbstractDispatcher.tryOptimizedDispatch(AbstractDispatcher.java:115)
    at org.springframework.integration.dispatcher.UnicastingDispatcher.doDispatch(UnicastingDispatcher.java:133)
    at org.springframework.integration.dispatcher.UnicastingDispatcher.dispatch(UnicastingDispatcher.java:106)
    at org.springframework.integration.channel.AbstractSubscribableChannel.doSend(AbstractSubscribableChannel.java:72)
    at org.springframework.integration.channel.AbstractMessageChannel.send(AbstractMessageChannel.java:570)

The invocation marked with * corresponds to the future.get(sendTimeout, TimeUnit.MILLISECONDS); call.

Seeing how the underlying client seems to support it (given by the fact that the future.get() call supports a timeout), how this be set? The only two properties I can find in the binder references (see here ) are spring.cloud.stream.kafka.binder.healthTimeout and batchTimeout , which as far as I can tell, do not influence this setting.

Seeing how the KafkaProducerMessageHandler is constructed in org.springframework.cloud.stream.binder.kafka.KafkaMessageChannelBinder.ProducerConfigurationMessageHandler , a private class, bean overridding is seemingly not the recommended way.

Answer 1

It doesn't appear to be documented, but similar to the listener container customizer https://docs.spring.io/spring-cloud-stream/docs/3.1.2/reference/html/spring-cloud-stream.html#_advanced_consumer_configuration you can add a ProducerMessageHandlerCustomizer @Bean to set arbitrary properties on the message handler.

In newer versions of the handler, the timeout is always configured to be at least as much as ProducerConfig.DELIVERY_TIMEOUT_MS_CONFIG , to avoid false negatives (where the publication is successful after the handler times it out).