In spring batch how can we move the processed files to another folder, I am using MultiResourceItemReader and chunk processing

Question

In Spring batch application how can we move files from BatchFileDir folder to a destination folder as each file gets processed. There would be a bunch of files in BatchFileDir folder so I am using MultiResourceItemReader. I am also following the chunk based processing.

I have all batch files as:

    @Value("BatchFileDir/batchFile_*.csv")
    private Resource[] allBatchFiles;

The Item reader is:

    @Bean
    public MultiResourceItemReader<MyServiceRequest> multiResourceItemReader() {
    MultiResourceItemReader<MyServiceRequest> resourceItemReader = new 
    MultiResourceItemReader<MyServiceRequest>();
    resourceItemReader.setResources(allBatchFiles);
    FlatFileItemReader<Record> reader = new FlatFileItemReader<>();
    DelimitedLineTokenizer tokenizer = new DelimitedLineTokenizer();
    String[] tokens = {"id", "name", "author", "subject"};
    tokenizer.setNames(tokens);
    DefaultLineMapper< MyServiceRequest> lineMapper = new DefaultLineMapper<>();
    lineMapper.setLineTokenizer(tokenizer);
    lineMapper.setFieldSetMapper(new RecordFieldSetMapper());
    reader.setLinesToSkip(1);
    reader.setLineMapper(lineMapper);
    resourceItemReader.setDelegate(reader);
    return resourceItemReader;
}

and the Item processor is:

    @Bean
    public ItemProcessor< MyServiceRequest, MyServiceResponse> itemProcessor() {
    return new ModifiedItemProcessor();
    }
   

the ModifiedItemProcessor is:

    public class ModifiedItemProcessor implements ItemProcessor< MyServiceRequest, 
    MyServiceResponse > {
    
    public MyServiceResponse process(MyServiceRequest item) {
    // interact with other Microservices and get the response
    return response;

the step is:

       @Bean
       protected Step step(@Qualifier("itemProcessor") ItemProcessor<MyServiceRequest, 
       MyServiceResponse> processor, ItemWriter<MyServiceRequest> writer) {
       return stepBuilderFactory
            .get("myStep")
            .<MyServiceRequest, MyServiceResponse> chunk(99)
            .reader(multiResourceItemReader())
            .processor(processor)
            .writer(writer)
            .build();
}

Answer 1

I don't see the added value of moving each file as soon as it gets processed, since all files will be moved after the step anyway. The listener approach should work, but there is an open issue on windows ( https://github.com/spring-projects/spring-batch/issues/1500 ). Hence, I recommend using a separate step to move files after the main step. For that, you can inject the same resources in that step and move files in a tasklet.