How to run Spring Batch Jobs in certain order (Spring Boot)?

Question

I'm developing with Spring Batch using Spring Boot.

I'm with the minimal configuration provided by Spring Boot and defined some Jobs (no XML configuration at all). But when I run the application,

SpringApplication.run(App.class, args);

the jobs are sequentially executed in some arbitrary order.

I'm defining the jobs this way in @Configuration annotated classes, Spring do the rest:

@Bean
public Job requestTickets() {
    return jobBuilderFactory.get(Config.JOB_REQUEST_TICKETS)
            .start(stepRequestTickets())
            .build();
}

How can I instruct the framework to run the jobs in a certain order?

EDIT: Could this warning give a hint? (Maybe has nothing to be)

2016-12-29 17:45:33.320  WARN 3528 --- [main] o.s.b.c.c.a.DefaultBatchConfigurer: No datasource was provided...using a Map based JobRepository

Answer 1

1.You first disable automatic job start by specifying spring.batch.job.enabled=false in application.properties

2.In your main class, do - ApplicationContext ctx = SpringApplication.run(SpringBatchMain.class, args); assuming your main class is named - SpringBatchMain.java.

This will initialize context without starting any jobs.

3.Once context is initialized, either you can do - JobLauncher jobLauncher = (JobLauncher) ctx.getBean("jobLauncher"); or do Autowired for this JobLauncher bean in main class and launch specific jobs sequentially in specific sequential order by invoking , jobLauncher.run(job, jobParameters) .

You can get specific job instances from context initialized at step # 2.

You can always use any ordered collection to put your jobs there and launch jobs by iterating over that collection.

4.This above technique works as long as your JobLauncher is configured to be synchronous ie main thread waits for jobLauncher.run() call to complete and that is default behavior of jobLauncher.

If you have defined your jobLauncher to use AsyncTaskExecutor then jobs will be started in parallel and sequential ordering will not be maintained.

Hope it helps !!

EDIT:

I was experimenting with @Order annotation as pointed by Stephane Nicoll and it seems to help only in creating an Ordered collection of jobs and that you can iterate and launch jobs in that order.

This below component gives me jobs in Order specified ,

@Component
public class MyJobs {
    @Autowired
    private List<Job> jobs;

    public List<Job> getJobs() {
        return jobs;
    }
}

and I can do , MyJobs myJobs = (MyJobs) ctx.getBean("myJobs"); in main class provided bean is defined,

@Bean
    public MyJobs myJobs() {
        return new MyJobs();
    }

I can iterate over myJobs and launch jobs in that order as specified by @Order annotation.

Answer 2

Order them.

@Bean
@Order(42)
public Job requestTickets() {
    return jobBuilderFactory.get(Config.JOB_REQUEST_TICKETS)
            .start(stepRequestTickets())
            .build();
}

See the javadoc of @Order for more details.

Answer 3

Here is an illustration of the solution.

This is so weird, it looks like we're hacking the process.

spring.batch.job.enabled=false

@SpringBootApplication
@EnableBatchProcessing
public class MyApplication {

    public static void main(String[] args)
            throws JobParametersInvalidException, JobExecutionAlreadyRunningException, JobRestartException, JobInstanceAlreadyCompleteException {

        ConfigurableApplicationContext ctx = SpringApplication.run(MyApplication.class, args);
        JobLauncher jobLauncher = (JobLauncher) ctx.getBean("jobLauncher");
        Job job1= (Job) ctx.getBean("job1");
        Job job2= (Job) ctx.getBean("job2");
        jobLauncher.run(job1,new JobParameters());
        jobLauncher.run(job2,new JobParameters());
    }

}

Answer 4

I don't have enough rep to comment. But have you tried just to manually launch your jobs in the order you want?

You need to set spring.batch.job.enabled=false in your application.properties, so that your jobs are not run automatically.

Then just use a launcher to launch your jobs in the order you want.

@RunWith(SpringRunner.class)
@SpringBootTest(classes = { TestConfiguration.class, TestDataSourceConfiguration.class, TestBatchConfig.class })
public class JobOrderTest {

    @Autowired
    JobLauncher jobLauncher;

    @Mock
    Job firstJob;

    @Mock
    Job secondJob;

    @Mock
    Job thirdJob;

    @Mock
    JobParametersValidator jobParametersValidator;

    @Test
    public void jobInOrderTest() throws JobExecutionAlreadyRunningException, JobRestartException, JobInstanceAlreadyCompleteException, JobParametersInvalidException {

        when(firstJob.getName()).thenReturn(UUID.randomUUID().toString());
        when(secondJob.getName()).thenReturn(UUID.randomUUID().toString());
        when(thirdJob.getName()).thenReturn(UUID.randomUUID().toString());
        when(firstJob.getJobParametersValidator()).thenReturn(jobParametersValidator);
        when(secondJob.getJobParametersValidator()).thenReturn(jobParametersValidator);
        when(thirdJob.getJobParametersValidator()).thenReturn(jobParametersValidator);

        jobLauncher.run(firstJob, new JobParameters());
        jobLauncher.run(secondJob, new JobParameters());
        jobLauncher.run(thirdJob, new JobParameters());
    }

}

Here is the output

2016-12-30 09:48:36.457  INFO 144860 --- [cTaskExecutor-1] o.s.b.c.l.support.SimpleJobLauncher      : Job: [firstJob] launched with the following parameters: ...
2016-12-30 09:48:36.457  INFO 144860 --- [cTaskExecutor-1] o.s.b.c.l.support.SimpleJobLauncher      : Job: [firstJob] completed with the following parameters: ...
2016-12-30 09:48:36.478  INFO 144860 --- [cTaskExecutor-2] o.s.b.c.l.support.SimpleJobLauncher      : Job: [secondJob] launched with the following parameters: ...
2016-12-30 09:48:36.478  INFO 144860 --- [cTaskExecutor-2] o.s.b.c.l.support.SimpleJobLauncher      : Job: [secondJob] completed with the following parameters: ...
2016-12-30 09:48:36.508  INFO 144860 --- [cTaskExecutor-3] o.s.b.c.l.support.SimpleJobLauncher      : Job: [thirdJob] launched with the following parameters: ...
2016-12-30 09:48:36.508  INFO 144860 --- [cTaskExecutor-3] o.s.b.c.l.support.SimpleJobLauncher      : Job: [thirdJob] completed with the following parameters: ...

Answer 5

if your one job is dependent on the second and so on, then do something like this.

@Configuration
@EnableBatchProcessing
@Import(DataSourceConfiguration.class)
public class AppConfig {

    @Autowired
    private JobBuilderFactory jobs;

    @Autowired
    private StepBuilderFactory steps;

    @Bean
    public Job job(@Qualifier("step1") Step step1, @Qualifier("step2") Step step2) {
        return jobs.get("myJob").start(step1).next(step2).build();
    }

    @Bean
    protected Step step1(ItemReader<Person> reader, ItemProcessor<Person, Person> processor, ItemWriter<Person> writer) {
        return steps.get("step1")
            .<Person, Person> chunk(10)
            .reader(reader)
            .processor(processor)
            .writer(writer)
            .build();
    }

    @Bean
    protected Step step2(Tasklet tasklet) {
        return steps.get("step2")
            .tasklet(tasklet)
            .build();
    }
}