A way to upsert multiple records together?

Question

I am trying to see if there is a way to improve the way data is inserted and updated.

I am using ORACLE DB with JDBC.

The current way i'm doing is to update (eg)customer record by using a FOR loop after checking if toUpdate is true . An Example such as the sample code below, followed by calling an existing DAO update() to do so. But this would not allow for the UPSERT of multiple data together.

However, is there a better way to UPSERT multiple data together?

        if (toUpdate) {
            for (Customer customerRec : customerRecList) 
                customerRecDAO.update(customerRec);
        }

Answer 1

Yes you can use batching:

public <T> int saveInBatch(List<T> records, String sql, Function<T, MapSqlParameterSource> paramFn){

try{
MapSqlParameterSource[] params = records.stream().map(paramFn).toArray(MapSqlParameterSource[]::new);
int rowCount = jdbcTemplate.batchUpdate(sql, params);
return Arrays.stream(rowCount).sum();
}
catch(Exception e){
//exception handling 
}
}

paramFn is a lambda of function such that you can map records to their values. example could be

(record)->{
return new MapSqlParameterSource("username" ,username),Integer.class);//just example
}

why we use MapSqlParameterSource

You can call saveInBatch in such a way that you pass smaller batches or customized batches of records. Suppose you have a million records then you may want to update only 200-400 records at a time so you can do something like below:

private <T> int saveRecords(List<T> records, String sql, Function<T, MapSqlParameterSource> paramFn) throws Exception{

return Lists.partition(records, 300).stream().map(batch-> saveInBatch(batch, sql, paramFn)).mapToInt(Integer::intValue).sum();
}

Note: above is not well optimized or streams are not used to their best but this is a working code I tried ages back:).