How to change ItemWriter target filename in a spring-batch job?

Question

I have a MultiResourceItemReader that loads multiple csv files from a folder.

For each file I'd like to change the filename that an ItemWriter is generating. But: how?

@Bean
public MultiResourceItemReader<String> reader() {
    FlatFileItemReader<String> delegate = new FlatFileItemReader<>();

    MultiResourceItemReader<String> r = new MultiResourceItemReader<>();
    r.setResources(new PathMatchingResourcePatternResolver().getResources("*.csv"));
    r.setDelegate(delegate);
    return r;
}

@Bean
public FlatFileItemWriter<String> writer() {
    FlatFileItemWriter<String> w = new FlatFileItemWriter<>();
    w.setResource(new FileSystemResource("dynamicfile.txt); //how dynamically, eg depending on the input filename?
    w.setLineAggregator(new PassThroughLineAggregator<>());
    return w;
}

Is it possible at all? And if not, how could I then write the data to different files using ItemWriter ?

Answer 1

You can't change the file name for a FlatFileItemReader instance because of the details around buffering, etc. However, there are two approaches you can use to accomplish the same thing.

ClassifierCompositeItemWriter
Using the ClassifierCompositeItemWriter , you'd create one FlatFileItemWriter for each file you want to write to. Then you'd create a Classifier implementation that chooses the reader based on the input item. The ClassifierCompositeItemWriter would use the Classifier you provide to choose which writer to use for each item.

You can read more about the ClassifierCompositeItemWriter in the documentation here: http://docs.spring.io/spring-batch/apidocs/org/springframework/batch/item/support/ClassifierCompositeItemWriter.html

Partitioning by resource
If the following conditions are met, you could use partitioning to handle the same thing:

1. Multiple input files
2. Multiple output files
3. The input files can be processed in parallel

In this case, implementing a Partitioner that is similar to the MultiResourcePartitioner , you can process each file independently (same logic as the MultiResourcePartitioner , just with the added attribute of the output file information as well). With a step scoped reader and writer, the input source and output destination could be configured dynamically.

You can read more about partitioning here: http://docs.spring.io/spring-batch/reference/html/scalability.html#partitioning