Spring Integration polling directories

Question

I have a use case where i need to upload .fif files from local to remote sftp server in a specified directory structure. Like following:

Now some other server/application will process those .fif files and will generate .npz files and will place it in the output folder of respective visit/id.

I am able to upload files using sftpOutboundAdapter and it is working all right.

Now, i cant figure out how to add polling to know that output files are created at this output directory of this visit. I dont want to download the files in output folder. I just want to know that there are 4 files created so that i can update the status of the visit.

The code I have so far is

@Value("${sftp.host}")
private String host;

@Value("${sftp.port:22}")
private int port;

@Value("${sftp.user}")
private String user;

@Value("${sftp.privateKey:#{null}}")
private Resource privateKey;

@Value("${sftp.privateKeyPassphrase:}")
private String privateKeyPassphrase;

@Value("${sftp.password:#{null}}")
private String password;

private final FileService fileService;

@Autowired
public SftpConfig(FileService fileService) {
    this.fileService = fileService;
}

@Bean
public SessionFactory<LsEntry> sftpSessionFactory() {
    DefaultSftpSessionFactory factory = new DefaultSftpSessionFactory(true);
    factory.setHost(host);
    factory.setPort(port);
    factory.setUser(user);
    if (privateKey != null) {
        factory.setPrivateKey(privateKey);
        factory.setPrivateKeyPassphrase(privateKeyPassphrase);
    } else {
        factory.setPassword(password);
    }
    factory.setAllowUnknownKeys(true);
    return new CachingSessionFactory<LsEntry>(factory);
}

@Bean
public IntegrationFlow sftpOutboundFlow() {
    return IntegrationFlows.from("toSftpChannel")
            .handle(Sftp.outboundAdapter(this.sftpSessionFactory(), FileExistsMode.FAIL)
                            .remoteDirectoryExpression("headers['path']")
                            .useTemporaryFileName(false)
                            .autoCreateDirectory(true)
                            .fileNameGenerator(message -> (String) message.getHeaders().get("fileName"))
                    , c -> c.advice(expressionAdvice())
            ).get();
}

@Bean
public Advice expressionAdvice() {
    ExpressionEvaluatingRequestHandlerAdvice advice = new ExpressionEvaluatingRequestHandlerAdvice();
    advice.setSuccessChannelName("integrationFlow.input");
    advice.setOnSuccessExpressionString("payload");
    advice.setFailureChannelName("integrationFlow.input");
    advice.setOnFailureExpressionString("payload");
    advice.setTrapException(true);
    return advice;
}

@Bean
public IntegrationFlow integrationFlow() {
    return f -> f.handle((MessageHandler) fileService::OnFilesUpload);
}

@MessagingGateway
public interface UploadGateway {

    @Gateway(requestChannel = "toSftpChannel")
    void upload(@Payload File file, @Header("fileName") String fileName, @Header("path") String path,
                @Header("parentId") Long parentId);

}

Please give java configuration of how can i poll the parent directory ie Providers and know the path till output folder of the respective visits/id in which the .npz files are created. and then get either the number of files that are created or the list of files.

Answer 1

Use the outbound gateway with the LS command (it has a recursive option if needed).

Simply send the remote directory to the gateway and it will return the list.

EDIT

To answer your comment below; you don't show your calling code, but why can't you do something like this...

@MessagingGateway
public interface UploadGateway {

    @Gateway(requestChannel = "toSftpChannel")
    void upload(@Payload File file, @Header("fileName") String fileName, @Header("path") String path,
            @Header("parentId") Long parentId);

    @Gateway(requestChannel = "toSftpLsGatewayChannel")
    List<LsEntry> getOutputFiles(String path);

}

and

this.gateway.upload(file, fileName, path, parentId);
List<LsEntry> outputs;
String outputPath = path.replace("/input", "/output");
do {
    outputs = this.gateway.getOutputFiles(outputPath);
    Thread.sleep(5_000);
} while (outputs == null || outputs.size() == 0);

If you want it to run asynchronously, just run the second part on a task executor.