keeping same chunk size for all jobs in Spring Batch Application

Question

Just wondering whether its a good practice to keep chunk sizes same for all jobs in a Spring Batch application or should we keep it different for different jobs, depending on the specific job's behaviour.

I can understand that obviously the answer depends upon a lot of factors, but just wanted to know whats the standard approach for this, if any.

Thanks

Answer 1

Chunk size is very specific to the job at hand. It's a key method of optimizing the performance of a batch job and will probably be specific to each job you write. For example, if you have small records, you may be able to have more in a chunk which would optimize the number of writes. Where if you have large records, you may not be able to fit as many in memory between writes.

Answer 2

I think understanding the factors to consider when deciding a chunk size will help answer this question. If these factors for example are same for all your jobs (though this is very rare) then why not.

Some books like "Spring Batch in Action" recommend to keep the chunk size commonly between 20 to 200. Some ideas from the same book are as follows:

1. Too small a chunk size creates too many transactions, which is costly and makes the job run slowly.
2. Too large a chunk size makes transactional resources—like databases—run slowly too, because a database must be able to roll back operations.
3. The best value for the commit interval depends on many factors: data, processing, nature of the resources, and so on.
4. The commit interval is a parameter in Spring Batch, so don't hesitate to change it to find the most appropriate value for your jobs.

So yes, the idea is to try running the jobs with different kind of sample data and different chunk sizes and compare for yourself the results and then choose the chunk size.