mule requester collection is not returning file name in it's properties

Question

When i am using mulerequester:request to fetch file from SFTP endpoint,filename comes under inbound property originalFileName .But when i tried to use mule-requester:request-collection filename did not come under inbound property originalFilename .

Can you guys help me out in fetching file names using mule-requester:request-collection ?

Answer 1

As you mentioned, mule-requester:request-collection returns a MuleMessageCollection so the originalFilename inbound property (and all others) should be in each MuleMessage of that collection. I'm pretty sure you can handle that collection with a foreach scope. HTH.

Answer 2

There were few bugs with mule-requester:request-collection ,
you can find it here :- Mule requester is consuming all the messages from JMS queue ... May be a bug
Make sure you haven't came across any such bugs with mule-requester:request-collection

Answer 3

I had an issue with losing the filename too. My initial flow as as follows

  <file:connector name="FormDefinitonFileConnector" autoDelete="true" streaming="false"
validateConnections="true" doc:name="File" fileAge="50" />
<flow name="migrateFromFilePath">
<mulerequester:request-collection resource="file:///C:/DEV" doc:name="Request a files from a location" />
<logger message="Message before foreach: #[message]" level="INFO" doc:name="Logger" />
<foreach doc:name="For Each" collection="#[payload]">
  <logger message="originalFilename: #[message.inboundProperties['originalFilename']]-#[server.dateTime]"
    level="INFO" doc:name="Logger" />
  <logger message="Message after foreach: #[message]" level="INFO" doc:name="Logger" />
  <file:outbound-endpoint path="C:/DEV/processed" responseTimeout="10000" doc:name="File" />
</foreach>

What I believe is happening, is the foreach is configured to loop through the array of the payload

<foreach doc:name="For Each" collection="#[payload]">

Where the contents of each payload is not a Mule message, but a collection of files that we have loaded.

INFO  2017-05-19 11:36:18,520 [[cf_migrator].api-
httpListenerConfig.worker.02] org.mule.api.processor.LoggerMessageProcessor: 
Message before foreach: 
org.mule.DefaultMessageCollection
{
id=fe1f27d0-3c7e-11e7-856a-54391dd0f207
payload=java.util.concurrent.CopyOnWriteArrayList
correlationId=<not set>    
correlationGroup=-1
correlationSeq=-1
encoding=UTF-8
exceptionPayload=<not set>

Message properties:
    INVOCATION scoped properties:
    _ApikitResponseTransformer_AcceptedHeaders=*/*
    _ApikitResponseTransformer_apikitRouterRequest=yes
    _ApikitResponseTransformer_contractMimeTypes=[]
    INBOUND scoped properties:
    OUTBOUND scoped properties:
    SESSION scoped properties:
    }
INFO  2017-05-19 11:36:18,569 [[cf_migrator].api-httpListenerConfig.worker.02] org.mule.api.processor.LoggerMessageProcessor: Setting originalFilename: null-2017-05-19T11:36:18.563+01:00
INFO  2017-05-19 11:36:18,570 [[cf_migrator].api-httpListenerConfig.worker.02] org.mule.api.processor.LoggerMessageProcessor: Message after foreach: 
org.mule.DefaultMuleMessage
{
  id=fe20d580-3c7e-11e7-856a-54391dd0f207
  payload=[B
  correlationId=fe1f27d0-3c7e-11e7-856a-54391dd0f207
  correlationGroup=2
  correlationSeq=1
  encoding=UTF-8
  exceptionPayload=<not set>

Message properties:
  INVOCATION scoped properties:
    _ApikitResponseTransformer_AcceptedHeaders=*/*
    _ApikitResponseTransformer_apikitRouterRequest=yes
    _ApikitResponseTransformer_contractMimeTypes=[]
    counter=1
    rootMessage=<<<MuleMessage>>>
  INBOUND scoped properties:
  OUTBOUND scoped properties:
    MULE_CORRELATION_GROUP_SIZE=2
    MULE_CORRELATION_ID=fe1f27d0-3c7e-11e7-856a-54391dd0f207
    MULE_CORRELATION_SEQUENCE=1
  SESSION scoped properties:
}

Thanks to this answer Use a splitter to retain filename

Replacing the foreach with a splitter, generates a Mule message for each file that is processed

<file:connector name="FormDefinitonFileConnector" autoDelete="true" streaming="false"
validateConnections="true" doc:name="File" fileAge="50" />
<flow name="migrateFromFilePath">
    <mulerequester:request-collection resource="file:///C:/DEV" doc:name="Request a files from a location" />
    <logger message="Message before splitter: #[message]" level="INFO" doc:name="Logger" />
      <collection-splitter doc:name="Collection Splitter"/>
      <logger message="Setting originalFilename: #[message.inboundProperties['originalFilename']]-#[server.dateTime]"
    level="INFO" doc:name="Logger" />
      <logger message="Message after splitter: #[message]" level="INFO" doc:name="Logger" />
      <file:outbound-endpoint path="C:/DEV/processed" responseTimeout="10000" doc:name="File" />
</flow>

Now we have a Mule message that contains the filename and the file contents as a payload

INFO  2017-05-19 11:50:07,240 [[cf_migrator].api-httpListenerConfig.worker.02] org.mule.api.processor.LoggerMessageProcessor:             
Message before splitter: 
org.mule.DefaultMessageCollection
{
  id=ec09b8b0-3c80-11e7-a8cf-54391dd0f207
  payload=java.util.concurrent.CopyOnWriteArrayList
  correlationId=<not set>
  correlationGroup=-1
  correlationSeq=-1
  encoding=UTF-8
  exceptionPayload=<not set>

Message properties:
  INVOCATION scoped properties:
    _ApikitResponseTransformer_AcceptedHeaders=*/*
    _ApikitResponseTransformer_apikitRouterRequest=yes
    _ApikitResponseTransformer_contractMimeTypes=[]
  INBOUND scoped properties:
  OUTBOUND scoped properties:
  SESSION scoped properties:
}
INFO  2017-05-19 11:50:07,286 [[cf_migrator].api-httpListenerConfig.worker.02] org.mule.api.processor.LoggerMessageProcessor: Setting originalFilename: collectionBetweenRowsFormDef.json-2017-05-19T11:50:07.281+01:00
INFO  2017-05-19 11:50:07,288 [[cf_migrator].api-httpListenerConfig.worker.02] org.mule.api.processor.LoggerMessageProcessor: Message after splitter: 
org.mule.DefaultMuleMessage
{
  id=ec0f5e00-3c80-11e7-a8cf-54391dd0f207
  payload=[B
  correlationId=ec09b8b0-3c80-11e7-a8cf-54391dd0f207
  correlationGroup=2
  correlationSeq=1
  encoding=UTF-8
  exceptionPayload=<not set>

Message properties:
  INVOCATION scoped properties:
    _ApikitResponseTransformer_AcceptedHeaders=*/*
    _ApikitResponseTransformer_apikitRouterRequest=yes
    _ApikitResponseTransformer_contractMimeTypes=[]
  INBOUND scoped properties:
    directory=C:\DEV
    fileSize=14696
    originalDirectory=C:\DEV
    originalFilename=file1.json
    timestamp=1495120341439
  OUTBOUND scoped properties:
    MULE_CORRELATION_GROUP_SIZE=2
    MULE_CORRELATION_ID=ec09b8b0-3c80-11e7-a8cf-54391dd0f207
    MULE_CORRELATION_SEQUENCE=1
  SESSION scoped properties:
}
INFO  2017-05-19 11:50:07,350 [[cf_migrator].api-httpListenerConfig.worker.02] org.mule.lifecycle.AbstractLifecycleManager: Initialising: 'FormDefinitonFileConnector.dispatcher.1900990765'. Object is: FileMessageDispatcher
INFO  2017-05-19 11:50:07,350 [[cf_migrator].api-httpListenerConfig.worker.02] org.mule.lifecycle.AbstractLifecycleManager: Starting: 'FormDefinitonFileConnector.dispatcher.1900990765'. Object is: FileMessageDispatcher
INFO  2017-05-19 11:50:07,354 [[cf_migrator].api-httpListenerConfig.worker.02] org.mule.transport.file.FileConnector: Writing file to: C:\DEV\processed\ec26b690-3c80-11e7-a8cf-54391dd0f207.dat
INFO  2017-05-19 11:50:07,357 [[cf_migrator].api-httpListenerConfig.worker.02] org.mule.api.processor.LoggerMessageProcessor:     Setting originalFilename: file1.json-2017-05-19T11:50:07.357+01:00
INFO  2017-05-19 11:50:07,357 [[cf_migrator].api-httpListenerConfig.worker.02] org.mule.api.processor.LoggerMessageProcessor:     Message after splitter: 
org.mule.DefaultMuleMessage
{
  id=ec0ffa40-3c80-11e7-a8cf-54391dd0f207
  payload=[B
  correlationId=ec09b8b0-3c80-11e7-a8cf-54391dd0f207
  correlationGroup=2
  correlationSeq=2
  encoding=UTF-8
  exceptionPayload=<not set>

Message properties:
  INVOCATION scoped properties:
    _ApikitResponseTransformer_AcceptedHeaders=*/*
    _ApikitResponseTransformer_apikitRouterRequest=yes
    _ApikitResponseTransformer_contractMimeTypes=[]
  INBOUND scoped properties:
    directory=C:\DEV
    fileSize=54143
    originalDirectory=C:\DEV
    originalFilename=file2.json
    timestamp=1495124035101
  OUTBOUND scoped properties:
    MULE_CORRELATION_GROUP_SIZE=2
    MULE_CORRELATION_ID=ec09b8b0-3c80-11e7-a8cf-54391dd0f207
    MULE_CORRELATION_SEQUENCE=2
  SESSION scoped properties:
}