Spring batch parallel processing create steps based on the step1 result

Question

I am trying to implement parallel processing using spring boot and spring batch. This batch will be triggered from UI with some required parameters

I need to create steps based on the request parameters, I tried as below,

The rest controller looks,

JobParameters jobParameters = new JobParametersBuilder().addLong("JobID",System.currentTimeMillis())
                    .addString("fileName", filename) 
                    .addString("buisinessDate", model.getGeneralServiceModel().getBusinessDate()) 
                    .addString("source", model.getGeneralServiceModel().getSource()) 
                    .toJobParameters();
            jobLauncher.run(job, jobParameters);

And the batch config:

Flow masterFlow = (Flow)new FlowBuilder("masterFlow").start(stepOne()).build();

    List<Step> steps = new ArrayList<Step>();
    for (ConcurrentLinkedQueue date : taskOne.readFile()) {
        steps.add(createStep(date));
    }

    return jobs.get("myJob")
            .start(masterFlow)
             .next(createParallelFlow(steps))
             .end()
             .build();

The masterFlow reads the job parameters into its variables, and the readFile() gives the list (based on this the steps has to be created), for this the jobParameters are required.

The problem is:

While starting my application itself the readFile() is getting executed. but I need to get it executed when job triggers through RestController since it has the required parameters.

How can I stop this execution while starting the application?

Answer 1

I need steps to be created based on step1 result

Creating steps is something that you do at configuration time. Step1 result can only be known at runtime. So in order to do this, you would need to access the application context at runtime and register step beans dynamically based on the result of Step1. I am not sure if this is really what you aim to do.

However, if you want to execute (not create, but execute) those steps based on step1's result, then you can use a JobExecutionDecider . Please refer to Programmatic Flow Decisions for more details and code example. A similar question can be found here: How to use decider in Spring batch?

But in order to process step1 I need request parameters.

I see you have managed to get request parameters and set them as job parameters. What you can do is access those job parameters in your step through the step execution. Here is an example if your step is a simple Tasklet:

class MyTasklet implements Tasklet {
    @Override
    public RepeatStatus execute(StepContribution contribution, ChunkContext chunkContext) throws Exception {
        Map<String, Object> jobParameters = chunkContext.getStepContext().getJobParameters();
        // use job parameters
        return RepeatStatus.FINISHED;
    }
}

If your step is a chunk-oriented tasklet, you can use a StepListener#beforeStep to get access to the step execution.

Hope this helps.