在第一个“找不到文件”问题之后，Sftp出站网关多个删除请求被卡住了

Question

I am trying to delete multiple files over Sftp and using the Spring Integration SftpOutboundGateway approach.

I am using QueueChannel for request and response. Also I am going Async way so that I can submit multiple request. I have also added an error channel recently.

Now the issue is that my first request is for a non-existing file, so I am fine to get a "2: No such file" exception message. But once that happens the other requests get stuck.

Following is the error message for the file not found case :

DEBUG o.s.integration.util.SimplePool - Obtained new org.springframework.integration.sftp.session.SftpSession@4ec427c0.
DEBUG o.s.i.f.r.s.CachingSessionFactory - Releasing Session org.springframework.integration.sftp.session.SftpSession@4ec427c0 back to the pool.
INFO  com.jcraft.jsch - Disconnecting from xxxx
DEBUG o.s.integration.util.SimplePool - Releasing org.springframework.integration.sftp.session.SftpSession@4ec427c0 back to the pool
INFO  com.jcraft.jsch - Caught an exception, leaving main loop due to Socket closed
Caused by: org.springframework.messaging.MessageHandlingException: error occurred in message handler [sftpDeleteFileHandler]; nested exception is org.springframework.messaging.MessagingException: Failed to execute on session; nested exception is org.springframework.core.NestedIOException: Failed to remove file.; nested exception is 2: No such file, failedMessage=GenericMessage [xxx]
    at org.springframework.integration.support.utils.IntegrationUtils.wrapInHandlingExceptionIfNecessary(IntegrationUtils.java:189)
    at org.springframework.integration.handler.AbstractMessageHandler.handleMessage(AbstractMessageHandler.java:186)
    at org.springframework.integration.endpoint.PollingConsumer.handleMessage(PollingConsumer.java:143)
    at org.springframework.integration.endpoint.AbstractPollingEndpoint.doPoll(AbstractPollingEndpoint.java:390)
    at org.springframework.integration.endpoint.AbstractPollingEndpoint.pollForMessage(AbstractPollingEndpoint.java:329)
    at org.springframework.integration.endpoint.AbstractPollingEndpoint.lambda$null$1(AbstractPollingEndpoint.java:277)
    at org.springframework.integration.util.ErrorHandlingTaskExecutor.lambda$execute$0(ErrorHandlingTaskExecutor.java:57)
    at org.springframework.core.task.SyncTaskExecutor.execute(SyncTaskExecutor.java:50)
    at org.springframework.integration.util.ErrorHandlingTaskExecutor.execute(ErrorHandlingTaskExecutor.java:55)
    at org.springframework.integration.endpoint.AbstractPollingEndpoint.lambda$createPoller$2(AbstractPollingEndpoint.java:274)
    at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54)
    at org.springframework.scheduling.concurrent.ReschedulingRunnable.run(ReschedulingRunnable.java:93)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    ... 1 more
Caused by: org.springframework.messaging.MessagingException: Failed to execute on session; nested exception is org.springframework.core.NestedIOException: Failed to remove file.; nested exception is 2: No such file
    at org.springframework.integration.file.remote.RemoteFileTemplate.execute(RemoteFileTemplate.java:446)
    at org.springframework.integration.file.remote.gateway.AbstractRemoteFileOutboundGateway.doRm(AbstractRemoteFileOutboundGateway.java:566)
    at org.springframework.integration.file.remote.gateway.AbstractRemoteFileOutboundGateway.handleRequestMessage(AbstractRemoteFileOutboundGateway.java:459)
    at org.springframework.integration.handler.AbstractReplyProducingMessageHandler.handleMessageInternal(AbstractReplyProducingMessageHandler.java:123)
    at org.springframework.integration.handler.AbstractMessageHandler.handleMessage(AbstractMessageHandler.java:169)
    ... 17 more
Caused by: org.springframework.core.NestedIOException: Failed to remove file.; nested exception is 2: No such file
    at org.springframework.integration.sftp.session.SftpSession.remove(SftpSession.java:83)
    at org.springframework.integration.file.remote.session.CachingSessionFactory$CachedSession.remove(CachingSessionFactory.java:225)
    at org.springframework.integration.file.remote.gateway.AbstractRemoteFileOutboundGateway.rm(AbstractRemoteFileOutboundGateway.java:586)
    at org.springframework.integration.file.remote.gateway.AbstractRemoteFileOutboundGateway.lambda$doRm$7(AbstractRemoteFileOutboundGateway.java:566)
    at org.springframework.integration.file.remote.RemoteFileTemplate.execute(RemoteFileTemplate.java:437)
    ... 21 more
Caused by: 2: No such file
    at com.jcraft.jsch.ChannelSftp.throwStatusError(ChannelSftp.java:2873)
    at com.jcraft.jsch.ChannelSftp.rm(ChannelSftp.java:1985)
    at org.springframework.integration.sftp.session.SftpSession.remove(SftpSession.java:79)
    ... 25 more

Update 1 :

I am using Spring Boot 2.1.8 -> Spring Integration 5.1.7

Config:

    @Bean
    public SessionFactory<LsEntry> sftpSessionFactory() {
        DefaultSftpSessionFactory factory = new DefaultSftpSessionFactory(true);
        factory.setHost(host);
        factory.setPort(port);
        factory.setUser(user);
        factory.setPassword(password);
        factory.setAllowUnknownKeys(true);
        return new CachingSessionFactory<LsEntry>(factory);
    }

    @Bean(name = PollerMetadata.DEFAULT_POLLER)
    public PollerMetadata defaultPoller() {

        PollerMetadata pollerMetadata = new PollerMetadata();
        pollerMetadata.setTrigger(new PeriodicTrigger(10));
        return pollerMetadata;
    }

    @Bean(name = "sftp.file.delete.request.channel")
    public MessageChannel sftpFileDeleteRequestChannel() {
        return new QueueChannel();
    }

    @Bean(name = "sftp.file.delete.response.channel")
    public MessageChannel sftpFileDeleteResponseChannel() {
        return new QueueChannel();
    }

    @Bean(name = "sftp.error.channel")
    public MessageChannel sftpErrorChannel() {
        return MessageChannels.queue("sftp.error.channel").get();
    }

    @Bean
    @ServiceActivator(inputChannel = "sftp.file.delete.request.channel", async = "true")
    public MessageHandler sftpDeleteFileHandler() {
        SftpOutboundGateway sftpOutboundGateway = new SftpOutboundGateway(sftpSessionFactory(), Command.RM.getCommand(),
                "headers['file_remoteDirectory'] + headers['file_remoteFile']");
        sftpOutboundGateway.setRequiresReply(true);
        return sftpOutboundGateway;
    }

    @ServiceActivator(inputChannel = "sftp.error.channel")
    public void sftpErrorHandler(final Message<MessageHandlingException> excpMessage) {
        log.error(excpMessage.getPayload().getCause());
    }

    @MessagingGateway(errorChannel = "sftp.error.channel")
    public interface SftpDeleteMessagingGateway {
        @Gateway(requestChannel = "sftp.file.delete.request.channel", replyChannel = "sftp.file.delete.response.channel")
        CompletableFuture<Message<Boolean>> deleteFile(final Message<Boolean> message);
    }

Code:

List<CompletableFuture<Message<Boolean>>> fileDeleteResults = new ArrayList<>();

foreach(...) {
        Message<Boolean> fileDeleteRequest = MessageBuilder.withPayload(true)
.setHeader(FileHeaders.REMOTE_DIRECTORY, directory)
.setHeader(FileHeaders.REMOTE_FILE, name).build();

 fileDeleteResults.add(sftpDeleteMessagingGateway.deleteFile(fileDeleteRequest));

}

try {
       CompletableFuture.allOf(fileDeleteResults.toArray(new CompletableFuture[fileDeleteResults.size()])).join();

       for (CompletableFuture<Message<Boolean>> fileDeleteResult : fileDeleteResults){

                Message<Boolean> message = fileDeleteResult.get();
                log.debug((String) message.getHeaders().get(FileHeaders.REMOTE_DIRECTORY)
                             + (String) message.getHeaders().get(FileHeaders.REMOTE_FILE) 
                             + ": " + message.getPayload());

                }
    } catch (CompletionException | InterruptedException | ExecutionException excp) {
            log.error(excp);
      }

Update 2 :

I modified the config as per the suggestion, but still facing the same issue. Below is the modified config -

    @Bean
    @ServiceActivator(inputChannel = "sftp.file.delete.request.channel")
    public MessageHandler sftpDeleteFileHandler() {
        SftpOutboundGateway sftpOutboundGateway = new SftpOutboundGateway(sftpSessionFactory(), Command.RM.getCommand(),
                "headers['file_remoteDirectory'] + headers['file_remoteFile']");
        sftpOutboundGateway.setRequiresReply(true);
        return sftpOutboundGateway;
    }

    @ServiceActivator(inputChannel = "sftp.error.channel")
    public boolean sftpErrorHandler(final Message<MessageHandlingException> excpMessage) {
        log.error(excpMessage.getPayload().getCause());
        return false;
    }

Answer 1

OK. Looks like your problem that you don't catch exceptions to go ahead.

See async option on the ServiceActivatingHandler :

/**
 * Allow async replies. If the handler reply is a {@link ListenableFuture}, send
 * the output when it is satisfied rather than sending the future as the result.
 * Ignored for return types other than {@link ListenableFuture}.
 * @param async true to allow.
 * @since 4.3
 */
public final void setAsync(boolean async) {

So, it is really an async only if a ListenableFuture (or Reactive Publsiher , though) is returned from the target handleRequestMessage() implementation. It is not a fact of the SftpOutboundGateway . Therefore your errorChannel = "sftp.error.channel" on the gateway definition is the right way to go. Although you need to return something from that sftpErrorHandler , which is going to be a return for the gateway call. Otherwise we are stuck waiting for reply or error.