删除集群中的已处理文件apache-camel

Question

I use apache camel to process files received on a ftp channel. My application is deployed in a cluster (4 nodes), for this I use RedisIdempotentRepository to ensure that a single node processes the file. My problem is that I want to delete the file after processing, if I use delete=true , the node A that processed the file when it finishes and will delete the file, node B already deleted it because the node B will not go through the filter and therefore it will directly access the deletion.

I would like to know how to only allow node A to delete the file?

from("sftp://host:port/folder?delete=true)
 .idempotentConsumer(simple("${file:onlyname}"),
     RedisIdempotentRepository.redisIdempotentRepository(redisTemplate, "camel-repo"))
 .bean("orderTrackingFileProcessor");

Answer 1

Configure the ftp endpoint to use the redis idempotent repository directly and not the idempotent consumer EIP in the route afterwards. That ensures only 1 ftp consumer is processing the same file.

If you have Camel in Action 2nd ed book its covered in the 2nd half of the transaction chapter.

Answer 2

I workaround this using pollEnrich :

adding delete step at end of processing:

.pollEnrich(remoteLocation + "?delete=true&fileName=${file:name}");

Full Example Route:

String remoteLocation = "sftp://host:port/folder";

from(remoteLocation)
 .idempotentConsumer(simple("${file:onlyname}"),
     RedisIdempotentRepository.redisIdempotentRepository(redisTemplate, "camel-repo"))
 .bean("orderTrackingFileProcessor")
 .pollEnrich(remoteLocation + "?delete=true&fileName=${file:name}");