How to use mule requester for SFTP - Dynamic file name and path

Question

I am trying to use a mule requester component for SFTP. How do I give the file name as part of the request argument? This is my flow:

<flow name="Move_Doc_To_Temp_Location" doc:name="Move_Doc_To_Temp_Location">
    <jdbc-ee:inbound-endpoint queryKey="Select_Document" queryTimeout="-1" pollingFrequency="30000" connector-ref="LoanApp_Database" doc:name="Select_Doc_To_be_moved"/>
    <set-variable variableName="OriginalPayload" value="#[payload]" doc:name="OriginalPayload"/>
    <logger message="#[payload['DocID']] - #[payload['Location']]" level="INFO" category="Document to be moved" doc:name="Logger"/>
           <!--  <mulerequester:request config-ref="Mule_Requester" resource="file:///#[payload['Location']]" doc:name="Mule Requester"/> -->
    <mulerequester:request config-ref="Mule_Requester" resource="sftp://user:Pwd@Hostname#[payload['Location']]" doc:name="Mule Requester"/>  
    <file:outbound-endpoint path="C:\Users\jvas\Desktop\testfiles" outputPattern="#[header:originalFilename]" responseTimeout="10000" doc:name="File"/>        
</flow>

The location value resolves to /opt/Documents/test.txt. When I specify the filename like this, it throws me an error:

ERROR 2014-07-24 05:51:12,358 [[loanapp_document_flow].Move_Doc_To_Temp_Location.stage1.02] org.mule.exception.DefaultMessagingExceptionStrategy: **************************************************************************** Message : Failed to receive event over: DefaultInboundEndpoint{endpointUri=sftp://muledev:@nylicvmmuledev/opt/Documents/test.txt, connector=SftpConnector { name=SFTP1 lifecycle=start this=aa033b numberOfConcurrentTransactedReceivers=4
createMultipleTransactedReceivers=true connected=true
supportedProtocols=[sftp] serviceOverrides= } , name='endpoint.sftp.nylicvmmuledev.opt.Documents.test.txt', mep=ONE_WAY, properties={}, transactionConfig=Transaction{factory=null, action=INDIFFERENT, timeout=0}, deleteUnacceptedMessages=false, initialState=started, responseTimeout=10000, endpointEncoding=UTF-8, disableTransportTransformer=false} (timeout=1000) Code
: MULE_ERROR-92 -------------------------------------------------------------------------------- Exception stack is: 1. Error 'No such file' occurred when trying to CDW to '/opt/Documents/test.txt'. (java.io.IOException)
org.mule.transport.sftp.SftpClient:102 (null) 2. Failed to receive event over: DefaultInboundEndpoint{endpointUri=sftp://muledev:@nylicvmmuledev/opt/Documents/test.txt, connector=SftpConnector { name=SFTP1 lifecycle=start this=aa033b numberOfConcurrentTransactedReceivers=4
createMultipleTransactedReceivers=true connected=true
supportedProtocols=[sftp] serviceOverrides= } , name='endpoint.sftp.nylicvmmuledev.opt.Documents.test.txt', mep=ONE_WAY, properties={}, transactionConfig=Transaction{factory=null, action=INDIFFERENT, timeout=0}, deleteUnacceptedMessages=false, initialState=started, responseTimeout=10000, endpointEncoding=UTF-8, disableTransportTransformer=false} (timeout=1000) (org.mule.api.transport.ReceiveException)
org.mule.transport.AbstractMessageRequester:132 ( http://www.mulesoft.org/docs/site/current3/apidocs/org/mule/api/transport/ReceiveException.html ) -------------------------------------------------------------------------------- Root Exception stack trace: java.io.IOException: Error 'No such file' occurred when trying to CDW to '/opt/Documents/test.txt'. at org.mule.transport.sftp.SftpClient.changeWorkingDirectory(SftpClient.java:102) at org.mule.transport.sftp.SftpConnector.createSftpClient(SftpConnector.java:196) at org.mule.transport.sftp.SftpConnector.createSftpClient(SftpConnector.java:174) + 3 more (set debug level logging or '-Dmule.verbose.exceptions=true' for everything)

---

Now I tried removing the file name from the location, and the SFTP was successful, but it has randomly chosen a file. The file has all permissions for SFTP and I am able to do the SFTP via command line successfully. I have referred to the documentation for the SFTP URL and this is what it provides for resolving the file name from the URL : http://tools.ietf.org/html/draft-ietf-secsh-scp-sftp-ssh-uri-04#section-4.1

Retrieve file.txt from the user's home directory on the host at host.example.com using SFTP using username user. This example assumes that the implementation supports the indication of a path relative to the home directory using a leading tilde.

    sftp://user@host.example.com/~/file.txt

Retrieve file.txt from the absolute path /dir/path on the host at host.example.com using SFTP using username user.

    sftp://user@host.example.com/dir/path/file.txt

Answer 1

The Mule SFTP connector can not take the filename in the address. Instead you need to use a filename filter. However, unfortunately I think the requestor module ignores the filters configured on the endpoint as mentioned here: http://forum.mulesoft.org/mulesoft/topics/mule_module_requester_add_support_for_filenamefilter

You could however use a script to read a file over sftp mid flow. Something like:

<scripting:transformer>
            <scripting:script engine="Groovy">
                <scripting:text>
                    def endpointBuilder = muleContext.endpointFactory.getEndpointBuilder( 
                    "sftp://${sftp.username}@${sftp.host}:${sftp.port}/${sftp.path}?identityFile=${app.home}/${sftp.keyPath}&amp;passphrase=${sftp.passphrase}&amp;connector=sftp-csv") 
                    endpointBuilder.addMessageProcessor(new org.mule.routing.MessageFilter(new org.mule.transport.file.filters.FilenameWildcardFilter(sessionVars.expectedFilename))) 
                    def inboundEndpoint = endpointBuilder.buildInboundEndpoint() 
                    inboundEndpoint.request(30000L) 
                </scripting:text>
            </scripting:script>
        </scripting:transformer>