Process all lines at the same time with spring batch

Question

I'm writing a batch program, using Spring batch, that reads from Db and then writes the information to a csv file. I've already used this framework before, but all the other programs that I wrote had the same structure:
1- read a line from db;
2- process the line individually;
3- write the line to the file;
But this new program has a different logic: I nedd to read lines from the Db, the I have to process all of them ( ex. if I found multiple lines with the same value in field A, I need to print only one line with field B that is the sum of the field B of the other lines ), and then I have to write. Is it possible to do something like this with Spring batch ? ( any example that I found works on single line ).
Thanks.

Answer 1

Instead of working with chunks (readers, writer, processors), Spring Batch can also work with Tasklets . A Tasklet is essentially one method, that does everything.

public class YourTasklet implements Tasklet {


    public RepeatStatus execute(StepContribution contribution,
                                ChunkContext chunkContext) throws Exception {


        // TODO do everything you want here 
        return RepeatStatus.FINISHED;
    }


}

You might want to read up on tasklets in the official documentation.

Answer 2

Aggregate functions require the entire data set to be processed in order to be calculated. The chunk-oriented processing model is not very friendly to implementing this kind of functions since data is processed in separate chunks.

In your case, since data is read from a database, I would leave the grouping/summing to the database with something like select sum(fieldB) ... group by fieldB . Databases are well optimized for this kind of tasks and you should end up reading a single record for each group.