batch processing with dynamic insert query using spring “jdbcTemplate.batchUpdate”

Question

Well I have been updating a legacy code since last few days.

Explanation : There is a table CUSTOMER with columns blah1, blah2, blah3, blah4, blah.....

As per our architecture I need to create an insert statement dynamically which can insert data into any table with any number of column.

Currently we have following code.

public void save(Table table, Connection conn) throws Exception {
    PreparedStatement pstmt = null;
    try {
        List<Row> rows = table.getRows();
        String sql = "";
        if(!rows.isEmpty() && rows != null)
        {
            for(Row row: rows) //READ EACH ROW
            {
                String columnName = ""; String columnValue = "";
                List<String> params = new ArrayList<String>();
                List<Column> columns = row.getColumns();

                if(!columns.isEmpty() && columns != null)
                {
                    for(Column column: columns) //GET EACH COLUMN DATA
                    {
                        columnName += ", "+column.getName();
                        columnValue += ", ?";

                        String value = column.getValue();
                        params.add(value); //ADD VALUE TO PARAMS
                    }
                    //INSERT QUERY
                    sql = "INSERT INTO "+table.getTableName()+" ("+columnName+") VALUES ("+columnValue+")";

                    if(pstmt == null) pstmt = conn.prepareStatement(sql);

                    //POPULATE PREPARED STATEMENT
                    for (int i =0; i<params.size(); i++) {
                        pstmt.setString(i+1, (String)params.get(i));
                    }
                    pstmt.addBatch();
                }
            }

            pstmt.executeBatch();//BATCH COMMIT
            conn.commit();
        }
    } catch (Exception e) {
        if (conn != null) {
            conn.rollback();
          }
        throw e;
    } 
}

Now instead of using the typical pstmt.executeBatch() . I want to use spring batch update as follows:

public void save(Table table, Connection conn) throws Exception{

         String sql = createSaveQuery(table);//CREATES the INSERT Query 

        getJdbcTemplate().batchUpdate(sql.toString(), new BatchPreparedStatementSetter() {

            @Override
            public void setValues(PreparedStatement ps, int j) throws SQLException {
                //PROBLEM AREA: How to map this for each insert statement?
                for(int i =0; i < params.size(); i++){
                    ps.setString(i+1, (String)params.get(i));
                }
            }
            @Override
            public int getBatchSize() {
                return 0;
            }
        });
    }

But I cannot figure out how to set the params for the each insert Query. As in the we can set the pstmt.setString(i, params.get(i)); for each row. How to achieve the same in 'new BatchPreparedStatementSetter()' .

any suggestions will be appreciated. If you need to further imporve the explanation. Please let me know.

Answer 1

I guess it's against Spring Batch nature to create dynamic SQL queries. That's because of PreparedStatement nature (read more here: What does it mean when I say Prepared statement is pre-compiled? ). Long story short - PreparedStatements are compiled on DB side and they are really fast. If you modify SQL query, it can't be reused. It is hard to give advise without knowing of actual DB schema, but in general you should have one POJO for CUSTOMER table (I hope you don't have BLOBS) and you should read-write all fields. It will be faster than "optimized" dynamic query.