在 Spring 批处理应用程序中使用 BufferWriter 处理回滚

Question

In my use case, i would like to use a buffer writer to handle the storing of strings.

• Only when the commit interval has been met, it will flush the writes

This is to ensure that when there is a error at reading, the buffer writer is able to handle the roll back.

Are there any examples or help on this, as it is my first time doing spring boot applications.

Thanks!

Answer 1

Unless you have a transactional file system, you cannot rollback a disk flush operation. Transactional file systems have never been mainstream because of the difficulty of the problem. Microsoft attempted to provide one , but this was abandoned quickly. There used be a couple of APIs for that as well, like Apache commons-transaction in the Java world, but those are not maintained anymore due to the lack of transactional file systems.

ACID compliant databases are the way to go to add transactional semantics on top of regular file systems. SQLite is the most widely used one, which is an amazing piece of engineering IMO. You can see how SQLite can be used as an application file format .

That's why Spring Batch cannot really rollback a flush operation, which brings me to the next point of how it handles rollbacks..

i would like to use a buffer writer to handle the storing of strings. Only when the commit interval has been met, it will flush the writes

This is already implemented in the FlatFileItemWriter provided by Spring Batch, which uses a TransactionawareBufferedWriter for that by default. This buffered writer is aware of the currently active transaction, and buffers items before flushing them to disk when the chunk size is met.

In the unlikely case where the transaction is rolled back after the buffer is flushed, the job will be marked as failed and you can restart it. On the restart, Spring Batch will truncate the corrupted file to the last known "good" byte offset, and restart a clean write of the last failed chunk, then continue from there.