Spring Integration SFTP channel adapter not sending the message to the down stream service activator for the second polling

Question

I am trying to implement simple SFTP channel adapter to accept files with any file name(to allow duplicate file names). For the first poll the file is transferred from SFTP server directory to the local directory and the service activator which is subscribed to this channel is able to receive the message but for the second time if i keep the file with same name in SFTP server, the file is transferred but the service activator was unable to get the message. I tried to implement local filter but it went into infinite loop (polling the file and creating the message).

configuration

<int:channel id="inboundMGetRecursive">
    <int:queue/>
</int:channel>

<int-sftp:inbound-channel-adapter id="sftpInboundAdapter"
                                  auto-startup="true"
                                  channel="channel1"
                                  session-factory="sftpSessionFactory"
                                  local-directory="c:/tmp/"
                                  remote-directory="${sftp.file.remote.inbound.dir}"
                                  auto-create-local-directory="false"
                                  filename-pattern="*.txt"
                                  local-filter="acceptAll">
    <int:poller fixed-delay="60000" error-channel="sftpExceptionsChannel"
                max-messages-per-poll="-1"/>
<bean id="acceptAll" class="org.springframework.integration.file.filters.AcceptAllFileListFilter"/
<int:service-activator id="serviceActivator" input-channel="channel1" ref="fileMoverHandler"
                       method="method1">
    <int:poller fixed-rate="3000" max-messages-per-poll="-1"/>
</int:service-activator>

Infinite loop logs

2016-12-14 13:25:39,632 [task-scheduler-1] [handlers.FileFilter] INFO  - Filter Activated
2016-12-14 13:25:39,635 [task-scheduler-1] [handlers.FileFilter] INFO  - file name : config.txt
2016-12-14 13:25:39,636 [task-scheduler-1] [file.FileReadingMessageSource] DEBUG - Added to queue: [C:\tmp\csvfiles\staging\config.txt]
2016-12-14 13:25:39,636 [task-scheduler-1] [file.FileReadingMessageSource] INFO  - Created message: [GenericMessage [payload=C:\tmp\csvfiles\staging\config.txt, headers={id=4a062556-4181-2806-d2ee-c0aa3221aec3, timestamp=1481743539636}]]
2016-12-14 13:25:39,636 [task-scheduler-1] [endpoint.SourcePollingChannelAdapter] DEBUG - Poll resulted in Message: GenericMessage [payload=C:\tmp\csvfiles\staging\config.txt, headers={id=4a062556-4181-2806-d2ee-c0aa3221aec3, timestamp=1481743539636}]
2016-12-14 13:25:39,636 [task-scheduler-1] [channel.QueueChannel] DEBUG - preSend on channel 'inboundMGetRecursive', message: GenericMessage [payload=C:\tmp\csvfiles\staging\config.txt, headers={id=4a062556-4181-2806-d2ee-c0aa3221aec3, timestamp=1481743539636}]
2016-12-14 13:25:39,636 [task-scheduler-1] [channel.QueueChannel] DEBUG - postSend (sent=true) on channel 'inboundMGetRecursive', message: GenericMessage [payload=C:\tmp\csvfiles\staging\config.txt, headers={id=4a062556-4181-2806-d2ee-c0aa3221aec3, timestamp=1481743539636}]
2016-12-14 13:25:39,637 [task-scheduler-1] [handlers.FileFilter] INFO  - Filter Activated
2016-12-14 13:25:39,637 [task-scheduler-1] [handlers.FileFilter] INFO  - file name : config.txt
2016-12-14 13:25:39,637 [task-scheduler-1] [file.FileReadingMessageSource] DEBUG - Added to queue: [C:\tmp\csvfiles\staging\config.txt]
2016-12-14 13:25:39,638 [task-scheduler-1] [file.FileReadingMessageSource] INFO  - Created message: [GenericMessage [payload=C:\tmp\csvfiles\staging\config.txt, headers={id=a2df4814-f759-0440-8ce2-c8d969a9315d, timestamp=1481743539638}]]
2016-12-14 13:25:39,638 [task-scheduler-1] [endpoint.SourcePollingChannelAdapter] DEBUG - Poll resulted in Message: GenericMessage [payload=C:\tmp\csvfiles\staging\config.txt, headers={id=a2df4814-f759-0440-8ce2-c8d969a9315d, timestamp=1481743539638}]
2016-12-14 13:25:39,638 [task-scheduler-1] [channel.QueueChannel] DEBUG - preSend on channel 'inboundMGetRecursive', message: GenericMessage [payload=C:\tmp\csvfiles\staging\config.txt, headers={id=a2df4814-f759-0440-8ce2-c8d969a9315d, timestamp=1481743539638}]
2016-12-14 13:25:39,639 [task-scheduler-1] [channel.QueueChannel] DEBUG - postSend (sent=true) on channel 'inboundMGetRecursive', message: GenericMessage [payload=C:\tmp\csvfiles\staging\config.txt, headers={id=a2df4814-f759-0440-8ce2-c8d969a9315d, timestamp=1481743539638}]
2016-12-14 13:25:39,639 [task-scheduler-1] [handlers.FileFilter] INFO  - Filter Activated
2016-12-14 13:25:39,639 [task-scheduler-1] [handlers.FileFilter] INFO  - file name : config.txt
2016-12-14 13:25:39,640 [task-scheduler-1] [file.FileReadingMessageSource] DEBUG - Added to queue: [C:\tmp\csvfiles\staging\config.txt]
2016-12-14 13:25:39,640 [task-scheduler-1] [file.FileReadingMessageSource] INFO  - Created message: [GenericMessage [payload=C:\tmp\csvfiles\staging\config.txt, headers={id=fd1ace38-6550-82ea-89c5-2a6229463167, timestamp=1481743539640}]]
2016-12-14 13:25:39,640 [task-scheduler-1] [endpoint.SourcePollingChannelAdapter] DEBUG - Poll resulted in Message: GenericMessage [payload=C:\tmp\csvfiles\staging\config.txt, headers={id=fd1ace38-6550-82ea-89c5-2a6229463167, timestamp=1481743539640}]
2016-12-14 13:25:39,640 [task-scheduler-1] [channel.QueueChannel] DEBUG - preSend on channel 'inboundMGetRecursive', message: GenericMessage [payload=C:\tmp\csvfiles\staging\config.txt, headers={id=fd1ace38-6550-82ea-89c5-2a6229463167, timestamp=1481743539640}]
2016-12-14 13:25:39,640 [task-scheduler-1] [channel.QueueChannel] DEBUG - postSend (sent=true) on channel 'inboundMGetRecursive', message: GenericMessage [payload=C:\tmp\csvfiles\staging\config.txt, headers={id=fd1ace38-6550-82ea-89c5-2a6229463167, timestamp=1481743539640}]

Please help me on how can i use filter to accept all files.

Answer 1

When using the accept all local filter, you need to remove/rename the file from the local disk when processing is complete, before the next poll.

Unless you also remove the remote file, you should also use an FtpPersistentAcceptOnceFileListFilter int the remote filter so the file is not re-fetched until its modified timestamp changes.