Spring Batch Transaction 使用基于块的处理

Question

enter image description hereI am using chunk based processing using Spring Batch to read data in chunks from DB using JdbcPagingItemReader. Now , I am killing the task in between the write stage of a chunk. Ideally the previous records in the chunk should have got rolled back but that did not happen. The DB used here is DB2 .

The approach which I used was- to set Autocommit false for the connection and then after the write steps were complete I used the commit statement.This approach worked fine for a small set of data. But in real time there would be millions of records.

So , is this the right approach and if not then what can be the other solutions.

Thanks!

Answer 1

If I'm understanding you correctly: do you want to do thousands of processings without committing. And when everything is done committing? Don't do so. You will have serious problems, both in the system and in the data base. Things like that you better use another strategy, for example: A temporary table. It goes on committing and when it finishes, it analyzes the result and if everything is right, it executes an update from the temporary to the final table.

You have to divide and conquer:

1. Run the process that generates the thousands of rows in a temporary table.
2. Analyze the result, it can be an analysis that results in the deletion of unsatisfactory lines or even an analysis that excludes the entire process.
3. Perform what should be done based on the analysis process.

I would create three or more Spring Batch Steps for each step of the one described above.