Update file after FlatFileItemReader in Spring Batch

Question

I currently have the following processing in a Spring Batch job:

• FlatFileItemReader reads a CSV file
• Processor does some work
• FlatFileItemWriter creates a mirror of the read file, but updates the file to reflect processing

I don't want to write to a new file, but I want to update the same file that is being read during processing.

My question is, is there a typical method in Spring to use the FlatFileItemReader and then update that same file per row in the processor at runtime?

Thanks for any help.

Answer 1

You can always write a custom writer in spring batch, just like an example below. Where you read data form the file into memory, and then update the same file with the data that you are intended to.

https://github.com/pkainulainen/spring-batch-examples/tree/master/spring/src/main/java/net/petrikainulainen/springbatch/custom/in

more than that FlatFileItemReader is not thread safe. Of course there are hacks to achieve thread safety but not a good practice to use such hacks, its always good to create custom writer.

Answer 2

Short answer is no, SB doesn't allow you to overwrite the same file you are reading from.

A better pratice is to write an intermediate file and then perform a delete/rename.

Write a temporary file is not a 'bad thing' especially if you are working with huge input file and OutOfMemoryException is round the corner; also using a temporary file can make your step restartable and allow you to manually retrive translated file also if delete/rename process fails.