MyBatis 3.2.7 bulky BATCH insert is blocked

Question

I'm running into an issue where I'm trying to insert BATCH of roughly 10,000 records into an Oracle 11g table of 13 columns. Actually this is blocked if I try more than 1000 records. Also, fine grained tests for a BATCH insert of 500 records is too slow as well.

The BATCH is generating huge trace files on the database sever due to ORA-4031 errors. The SQL statement running in each session is over 80,000 lines long, taking as input over 42,000 bind variables.

Driver: oracle.jdbc.driver.OracleDriver

I'm trying to satisfy the requirement: If any data for an insert is invalid, I got to rollback.

My question is:

Is there a way to tell MyBatis to divide the incoming bulky batch into small batches say 10 records each, insert them but rollback all the batches if one of the batch fails and commit only when all the batches are sucessfully inserted?

Or is there an alternative approach to this?

Below is the xml mapping for the insert:

<insert id="insertToTable" parameterType="java.util.List">
        INSERT ALL
        <foreach item="line" collection="list" >
            <foreach item="lineItem"
                collection="line.entrySet()" open="" close="" separator="">
                into
                TABLE_TEST_T
                (col1, col2, col3, col4, col5,
                col6, col7, col8, col9, col10,
                col11, col12, col13)
                values(
        <!--        #{lineItem.item1, jdbcType=DATE}, -->
                #{lineItem.item1, jdbcType=VARCHAR},
                #{lineItem.item2, jdbcType=VARCHAR},
                #{lineItem.item3, jdbcType=VARCHAR},
                #{lineItem.item4, jdbcType=NUMERIC},
                #{lineItem.item5, jdbcType=NUMERIC},
                #{lineItem.item6, jdbcType=NUMERIC},
                #{lineItem.item7, jdbcType=NUMERIC},
                #{lineItem.item8, jdbcType=NUMERIC},
                #{lineItem.item9, jdbcType=NUMERIC},
                #{lineItem.item10, jdbcType=NUMERIC},
                #{lineItem.item11, jdbcType=NUMERIC},
                #{lineItem.item12, jdbcType=NUMERIC},
                #{lineItem.item13, jdbcType=NUMERIC}    
                )           
            </foreach>
        </foreach>
        SELECT * FROM dual
    </insert>

Appreciate your feedback.

Answer 1

Yes, you can start a transaction and roll it back if error happens or commit it if there are no errors. There are many ways to manage transactions in Java applications

Since you are using myBatis, you can look into SqlMapTransactionManager .

Answer 2

I used a camel EIP Splitter in a route and tied the query in a transaction. More details can be found at http://camel.465427.n5.nabble.com/Camel-Mybatis-2-13-1-BATCH-of-10-000-records-tt5756211.html