Mule Requester - Picking same file

Question

While I am trying to move two files (say File1.txt and File.txt) with the help of MuleRequester. MuleRequester is picking File1 two times. Rather than it should first pick File1 and then File2. Can anyone please help me provide the resolution? I am using 3.8.0 runtime. have attached snapshot of flow for your ready reference

Note : Http requester has been used just to make execution user driven.

Answer 1

Whenever using File/SFTP/FTP component should be more careful on polling and FileAge( If needed for long size). Based on your Size of the payload, we need to increase the polling frequency.If the file size is large and the polling freq is less, it might read multiple times which end up in causing issues.

Also make sure your file is deleted once the file is read.

Below config working fine.

 <file:connector name="FileCon" autoDelete="true" streaming="false" validateConnections="true" doc:name="File" fileAge="50" pollingFrequency="2000"/>
<flow name="removeFlow">
    <http:listener  config-ref="HTTP_Listener_Configuration"  path="/" doc:name="HTTP"/>
    <set-payload value="#['test']" doc:name="Set Payload"/>
    <async doc:name="Async">
        <logger level="INFO" doc:name="Logger"/>
        <mulerequester:request-collection config-ref="Mule_Requester" resource="file://c:/in?connector=FileCon" timeout="30000" count="2" doc:name="Mule Requester" throwExceptionOnTimeout="true" />
        <logger level="INFO" doc:name="Logger"/>
        <foreach doc:name="For Each">
            <file:outbound-endpoint path="c:/out" outputPattern="#[server.nanoTime()]#[function:uuid].txt" responseTimeout="10000" doc:name="File"/>
        </foreach>
    </async>
</flow>

Answer 2

The requester does not on it own cycle through files in an SFTP path. If you do not remove the first file from the path on the second access it will again get the same file. Further, if you attempt to get a collection of files, it will get the same file over and over. That second point, IMO is a flaw in the requester and should be corrected. In any case, the result is you cannot get to the second file in the directory with the requester without consuming the first file to remove it from the path or if using a wildcard or regex filter renaming the file so it no longer matches you request criteria.

An option is to obtain a list of files and then build your request URL from the list of files, but this requires you to separately access the SFTP server to get this list, or simply know the names of the files you are looking for before you try to request them.