在面向读写块的过程中，Spring Batch 在哪里提交？

Question

On my java project, I am having a read ( FlatFileItemReader ) - write ( JdbcBatchItemWriter ) chunk oriented process using Spring boot. Using HikariCp datasources.

Looking into the Spring batch jdbc writer we can find that part :

               //some code
                @Override
                public int[] doInPreparedStatement(PreparedStatement ps) throws SQLException, DataAccessException {
                    for (T item : items) {
                        itemPreparedStatementSetter.setValues(item, ps);
                        ps.addBatch();
                    }
                    return ps.executeBatch();
                }
              //some code

So basically here if my Hikari datasource has setAutocommit parameter to true it means that after ps.executeBatch() my database will be updated.

First question here, on PreparedStatement.executeBatch() how does it process exactly? Does it commit every sql statement or at the end of every sql statements?

In the case where setAutocommit parameter is on false , after ps.executeBatch() it should not be updated.

Since Spring batch should manage the transaction and so the commit. I tried to find where does it commit for a better understanding on an issue that I have. Looking to ChunkOrientedTasklet , SimpleChunkProcessor and TransactionTemplate I could not find where in the process it does the commit.

So my second question is, where does Spring batch exactly commit on a chunk oriented process?

EDIT : Using Spring batch 3.0.7

EDIT : It seems processed in AbstractPlatformTransactionManager.processCommit(DefaultTransactionStatus status) but still can not understand how.

Answer 1

Your autocommit parameter is irrelevant, Spring will manage things and override that as it sees fit (and it does see it fit).

You can set the commit interval to define how often you want to commit.

Whether you should change it depends entirely on how you intend to batch, whether your execution can fail, whether those cases should be skipped or retried and other such things.

Answer 2

Regarding your question on "when is it committed?", per the Spring Batch docs :

5.1 Chunk-Oriented Processing

Spring Batch uses a 'Chunk Oriented' processing style within its most common implementation. Chunk oriented processing refers to reading the data one at a time, and creating 'chunks' that will be written out, within a transaction boundary. One item is read in from an ItemReader, handed to an ItemProcessor, and aggregated. Once the number of items read equals the commit interval, the entire chunk is written out via the ItemWriter, and then the transaction is committed.

Below is a code representation of the same concepts shown above:

 List items = new Arraylist(); for(int i = 0; i < commitInterval; i++){ Object item = itemReader.read() Object processedItem = itemProcessor.process(item); items.add(processedItem); } itemWriter.write(items);

Hence, if you're going to use Spring Batch, you shouldn't be messing with setting auto-commit, as this is something you should trust the framework to handle for you. The section immediately below that shows an example of how this is configured:

5.1.1 Configuring a Step

Despite the relatively short list of required dependencies for a Step, it is an extremely complex class that can potentially contain many collaborators. In order to ease configuration, the Spring Batch namespace can be used:

 <job id="sampleJob" job-repository="jobRepository"> <step id="step1"> <tasklet transaction-manager="transactionManager"> <chunk reader="itemReader" writer="itemWriter" commit-interval="10"/> </tasklet> </step> </job>

The configuration above represents the only required dependencies to create an item-oriented step:

• reader - The ItemReader that provides items for processing.

• writer - The ItemWriter that processes the items provided by the ItemReader.

• transaction-manager - Spring's PlatformTransactionManager that will be used to begin and commit transactions during processing.

• job-repository - The JobRepository that will be used to periodically store the StepExecution and ExecutionContext during processing (just before committing). For an in-line (one defined within a ) it is an attribute on the element; for a standalone step, it is defined as an attribute of the .

• commit-interval - The number of items that will be processed before the transaction is committed.