Spring batch with remote partitioning dynamic worker name

Question

I am creating a batch template project which is used by many other applications to implement batch processing with partitioning. I have implemented master and worker configurations in a generic way so that these can be reused by all the applications. I want to use the worker step name from the property file and use it dynamically in partitioner step as below.

@Bean
public Step partitionerStep() {
        return stepBuilderFactory.get(configs.getPartitionStepName())
                .partitioner("worker", partitioner(null))
                .outputChannel(outboundRequests())
                .inputChannel(inboundRequests())
                .build();
    }

Worker configuration as below

@Bean
    public Step worker(ItemWriter<T> itemWriter) {
        return stepBuilderFactory.get("worker")
                .inputChannel(inboundRequests())
                .outputChannel(outboundRequests())
                .<T, T>chunk(100)
                .reader(itemReader(null))
                .processor(itemProcessor())
                .writer(itemWriter)
                .build();
    }

I found that the worker step name in master step has to be same as the worker step bean name otherwise it's throwing bean not found error.

Is there a way to change this to use the worker step name instead of worker step bean name?

Answer 1

The bean name and the step name can be different. For example:

@Bean
public Step myWorkerStep(ItemWriter<T> itemWriter) {
    return stepBuilderFactory.get("worker")
                .inputChannel(inboundRequests())
                .outputChannel(outboundRequests())
                .<T, T>chunk(100)
                .reader(itemReader(null))
                .processor(itemProcessor())
                .writer(itemWriter)
                .build();
}

In this sample, the bean name is myWorkerStep , while the step name is worker .

Is there a way to change this to use the worker step name instead of worker step bean name?

Yes, you just set the name as needed in both places, ie on the manager side .partitioner("worker", partitioner(null)) and on the worker side return stepBuilderFactory.get("worker") .