Groovy SQL eachRow format output

Question

I have to synchronize MSSQL DB with MYSQL DB. (>500 Tables and total >10000000 lines).
I have a working solution, but the performance is not acceptable.
The bottleneck may be here:

I read from the MSSQL database with

mssqlDaten = mssql.eachRow (....) { row_data -> ...

because of the changing field names in the different tables i'm using a for loop to create the insert statement

    str = "insert into current_table values ("
    for (over all fields from the current table) {
    str += "'" + row_data[i] + "', "
    } 
    str += ")" 
    mssql.execute (str)

This may be the problem for every line i have to go thru the loop.

Has someone an idea to get the row-data directly in the format (ValueField1, ValueField2,....). When i print the row data now, i get (NameField1:ValueField1, NameField2.ValueField2,...).

Answer 1

try to do insertion with batch like this:

def table = "MYTABLE"
//the list of column names could be selected from source database
def columns = ["A","B","C","D"]

def sQuery = "select ${columns.join(',')} from $table".toString()

//build insert query with column list and ? as value placeholders
//as result we have query like : insert into MYTABLE ( A,B,C,D ) values ( ?,?,?,? )
def iQuery = "insert into $table ( ${columns.join(',')} ) values ( ${ columns.collect{'?'}.join(',') } )".toString()

def updateCounts = mysql.withBatch(100, iQuery) { ps ->
    mssql.eachRow (sQuery) { row_data ->
        def row_array = columns.collect{ colname-> rowdata[colname] }
        ps.addBatch( row_array )
    }
}