如何将 arguments 从从步骤传递到 Spring 批处理中的读卡器？

Question

I have a spring batch process that is reading data from a database. Basically what happens is I have a SQL query that needs to get data by a column (type) value. That column has 50 different values. So there are 50 queries and each is executed on a separate slave step. But the query is building inside the Reader. So I need to pass each type to the Reader to build the query and read data. I am using Partitioner to separate the query with Offset and Limit .

Here is the code I have,

private Flow flow(List<Step> steps) {
    SimpleAsyncTaskExecutor taskExecutor = new SimpleAsyncTaskExecutor();
    taskExecutor.setConcurrencyLimit(1);

    return new FlowBuilder<SimpleFlow>("flow")
            .split(taskExecutor).add(steps.stream().map(step -> new FlowBuilder<Flow>("flow_" + step.getName())
                    .start(step).build()).toArray(Flow[]::new)).build();
}

@Bean
public Job job() {
    List<Step> masterSteps = TYPES.stream().map(this::masterStep).collect(Collectors.toList());
    return jobBuilderFactory.get("job")
            .incrementer(new RunIdIncrementer())
            .start(flow(masterSteps))
            .end()
            .build();
}

@Bean
@SneakyThrows
public Step slaveStep(String type) {
    return stepBuilderFactory.get("slaveStep")
            .<User, User>chunk(100)
            .reader(reader(type, 0, 0))
            .writer(writer())
            .build();
}

@Bean
@SneakyThrows
public Step masterStep(String type) {
    return stepBuilderFactory.get("masterStep")
            .partitioner(slaveStep(type).getName(), partitioner(0))
            .step(slaveStep(type))
            .gridSize(5)
            .taskExecutor(executor)
            .build();
}

@Bean
@StepScope
@SneakyThrows
public JdbcCursorItemReader<User> reader(String type,
                                        @Value("#{stepExecutionContext['offset']}") Integer offset,
                                        @Value("#{stepExecutionContext['limit']}") Integer limit) {
    String query = MessageFormat.format(SELECT_QUERY, type, offset, limit); // Ex: SELECT * FROM users WHERE type = 'type' OFFSET 500 LIMIT 1000;

    JdbcCursorItemReader<User> itemReader = new JdbcCursorItemReader<>();
    itemReader.setSql(query);
    itemReader.setDataSource(dataSource);
    itemReader.setRowMapper(new UserMapper());
    itemReader.afterPropertiesSet();

    return itemReader;
}

@Bean
@StepScope
public ItemWriter<User> writer() {
    return new Writer();
}

@Bean
@StepScope
public Partitioner partitioner(@Value("#{jobParameters['limit']}") int limit) {
    return new Partitioner(limit);
}

The issue I am using is to reader() method the type value is not passing. And even when I am adding @Bean annotation it is saying Could not autowire. No beans of 'String' type found. Could not autowire. No beans of 'String' type found. . If I didn't put @Bean offset and limit is always 0 because @Value is not populating. Right now when I am executing the batch nothing happens inside reader because type is null. When I am hardcoding the value it is working. So how can I fix this? Thanks in advance.

Answer 1

If you are iterating every TYPE and execute masterStep , why don't you remove that TYPE logic and instead you can SELECT * FROM table OFFSET? LIMIT? SELECT * FROM table OFFSET? LIMIT? and handle offset and limit inside Partitioner ? And then your 5 threads will handle this. If your final goal is to process every record in that table then you can simply use this without worrying about TYPE and executing them inside separate Step.