Increase performance in Mule ESB having multiple for each and multiple flow reference

Question

We need to process millions of records, from database to microsoft product.

In our flow we are calling multiple flows and in each flow we are using for each. And we need sequential process because we are using one flow response in another flow request.

We don't need to use batch processing.

Is there any way to increase performance and process this flow faster

Answer 1

The best way to speed up a process like this is to make parts of it asynchronous. The way I would do it is to read each input record (using streaming capability of Mule Database component) and put it onto a JMS queue. You then have another flow that reads from the queue and processes that record, calling each processing flow sequentially for each record, but processing multiple records at a time. The throughput can then be configured with your JMS connection settings.

If the processing of one record depends on the result of processing the previous record, then you are stuck with the current sequential processing, so you would need to re-architect the solution and try to remove that requirement.