Spring Batch Partitioning : Slave step with multiple steps

Question

Given a Spring Batch Job that uses partitioning, is it possible to have more than one partition step ?

For example:

<batch:step id="masterStep"
    next="nextStep">
         <partition step="slaveStep1"
              partitioner="dataPartitioner">
                <batch:handler grid-size="10" task-executor="taskExecutor" />
         </partition>
</batch:step>

In the above example, is it possible to add another parition step to the masterStep (preferably without needing to provide a partitioner for each partitioned step)? If not, is there another way to configure multiple steps that will execute one after another for each partition?

Answer 1

is it possible to have more than one partition step?

Yes, you can have multiple partitioned steps in a single job.

In the above example, is it possible to add another parition step to the masterStep (preferably without needing to provide a partitioner for each partitioned step)?

This means you will have "nested" partitioned steps: a partitioned step for which each worker step is in turn a partitioned step. The XSD does not allow you to do that, but you can probably do it with Java config (I never tried that, but IMO it will make things very complex).

If not, is there another way to configure multiple steps that will execute one after another for each partition?

You can try to use a JobStep for that. The idea is to have a job with one paritioned step for which each worker step is a JobStep . Now each worker is a job with multiple regular steps (the steps you want to run for each partition).