mule : Scatter Gather : How to aggregate variables from different routes if we are setting a target variable in the Scatter gather component?

Question

Using Mule 4, Scatter Gather component, I set the target variable as "result". I have two more variables

1. firstVar: Defined outside the Scatter Gather component but updated inside one route
2. secondVariable: Defined inside the Scatter Gather component.

When I run the code, I expect to see firstVar (updated), secondVariable and result in the vars when Scatter Gather is finished. But I see only firstVar with the initial value.

Observation: If i remove the target variable result, I see the firstVar (updated), secondVariable and the aggregated payload as well.

Can someone please explain what am I doing wrong?

Code for reference:

<?xml version="1.0" encoding="UTF-8"?>

<mule xmlns:ee="http://www.mulesoft.org/schema/mule/ee/core"
    xmlns:http="http://www.mulesoft.org/schema/mule/http"
    xmlns="http://www.mulesoft.org/schema/mule/core"
    xmlns:doc="http://www.mulesoft.org/schema/mule/documentation"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://www.mulesoft.org/schema/mule/core http://www.mulesoft.org/schema/mule/core/current/mule.xsd
http://www.mulesoft.org/schema/mule/http http://www.mulesoft.org/schema/mule/http/current/mule-http.xsd
http://www.mulesoft.org/schema/mule/ee/core http://www.mulesoft.org/schema/mule/ee/core/current/mule-ee.xsd">
    <flow name="ScatterGather-ExampleFlow"
        doc:id="5dec6c32-4da7-4be0-91e1-26744e6e4761">
        <http:listener doc:name="Listener"
            doc:id="a75d1ddb-0481-45be-992d-39e397160257"
            config-ref="HTTP_Listener_config" path="/api/scattergather" />
        <logger level="INFO" doc:name="Logger"
            doc:id="9bb44a1f-287f-415c-b19f-87798797cfa8"
            message="Starting Scatter Gather Sample" />
        <set-variable value='#["First Value"]'
            doc:name="First Variable"
            doc:id="1eb7430b-f6a8-4366-8594-8f76479148ab" variableName="firstVar" />
        <scatter-gather doc:name="Scatter-Gather"
            doc:id="ef921649-8269-46ba-b38e-8e55f2694368" target="result">
            <route>
                <ee:transform doc:name="Transform Message"
                    doc:id="c5b2717c-5917-4641-aa4c-2283614cc4af">
                    <ee:message>
                        <ee:set-payload><![CDATA[%dw 2.0 output application/java 
            --- payload]]></ee:set-payload>
                    </ee:message>
                    <ee:variables>
                        <ee:set-variable variableName="firstVar"><![CDATA[%dw 2.0 output application/java --- 
            "updated First Value"]]></ee:set-variable>
                    </ee:variables>
                </ee:transform>
                <set-payload value='#["First route PAyload"]'
                    doc:name="Set Payload"
                    doc:id="8c479443-dd94-410e-9346-fedb7c0d3d71" />
            </route>
            <route>
                <set-variable value='"secondValue"'
                    doc:name="SecondVariable"
                    doc:id="6f78c3a7-ae00-4693-93ca-da98217d36d4"
                    variableName="secondVariable" />
                <set-payload value='#["2nd routePayload"]'
                    doc:name="Set Payload"
                    doc:id="aad2f059-2a35-4c29-85d7-45e33f317e59" />
            </route>
        </scatter-gather>
        <logger level="INFO" doc:name="Logger"
            doc:id="d03caeac-41e3-4dc9-8b67-de1da841011c" message="#[vars]" />
    </flow>
</mule>

Answer 1

I guess it is related to the event being discarded by the assignment to the target variable. That is probably to be expected. The event contains both the message/payload and the variables. Using target only assigns the resulting payload to the variable. The rest of the modified event is lost.

I would recommend to use a payload that stores all the needed data as additional attributes. You can later use DataWeave to extract the attributes from the rest of the payload.

Answer 2

Without the target variable, after all processing routes have finished execution, the Scatter-Gather component creates a new Mule event that combines all resulting Mule events from each route, and then passes the new Mule event to the next component in the flow.

If Scatter Gather ->General->target is used then it populates the resultant payload (multiple mule events ) into a new target variable and the original mule event and its vars are maintained.