Java MySQL批量插入与普通

Question

我试图更快地插入mysql，所以我使用批量插入重新编写代码。 问题是，当我运行以下代码时，两种方法的结果相同 （时间已过去：2分钟或173秒）。 对此事的任何评论都是受欢迎的。

MySQLConn mysqlconn = new MySQLConn(db,username,pass);
    Connection conn =mysqlconn.getConnection();
    PreparedStatement ps = null;

    String query = "insert into table (column) values (?)";

    System.out.println("bulk insert started with "+10000 );
    long startTime = System.currentTimeMillis();

    try {
        ps = conn.prepareStatement(query);
        long start = System.currentTimeMillis();
        for(int i =0; i<10000;i++){

            ps.setString(1, "Name"+i);

            ps.addBatch();

            if(i%1000 == 0) ps.executeBatch();
        }
        ps.executeBatch();

        System.out.println("Time Taken="+(System.currentTimeMillis()-start));

    } catch (SQLException e) {
        e.printStackTrace();
    }

    System.out.println("End inserting data...");

    long stopTime = System.currentTimeMillis();
    long elapsedTime = (stopTime - startTime); 

    long days = TimeUnit.MILLISECONDS.toDays(elapsedTime);
    long hours = TimeUnit.MILLISECONDS.toHours(elapsedTime);
    long minutes = TimeUnit.MILLISECONDS.toMinutes(elapsedTime);
    long seconds = TimeUnit.MILLISECONDS.toSeconds(elapsedTime);


    System.out.println("Time elapsed: "+minutes+"min, or "+seconds+"sec");



    System.out.println("normal insert started with "+10000 );

    long startTimeNormal = System.currentTimeMillis();

    for(int i=0;i<10000;i++){ 
        String ins="Data"+i;
        Insert insert = new Insert(conn,ins,"db.table", "column" );
    }
    System.out.println("End inserting data...");

    long stopTimeNormal = System.currentTimeMillis();
    long elapsedTimeNormal = (stopTimeNormal - startTimeNormal); 

    long daysN = TimeUnit.MILLISECONDS.toDays(elapsedTime);
    long hoursN = TimeUnit.MILLISECONDS.toHours(elapsedTime);
    long minutesN = TimeUnit.MILLISECONDS.toMinutes(elapsedTime);
    long secondsN = TimeUnit.MILLISECONDS.toSeconds(elapsedTime);


    System.out.println("Time elapsed: "+minutesN+"min, or "+secondsN+"sec");

Answer 1

您使用addBatch和executeBatch ，这很好，但您还需要将连接上的自动提交设置为false，以便实现更快的执行时间。

一些事情：

Connection conn = mysqlconn.getConnection();
conn.setAutoCommit(false); // here

PreparedStatement ps = null;

String query = "insert into table (column) values (?)";

System.out.println("bulk insert started with "+10000 );
long startTime = System.currentTimeMillis();

try {
    ps = conn.prepareStatement(query);
    long start = System.currentTimeMillis();
    for(int i =0; i<10000;i++){

        ps.setString(1, "Name"+i);

        ps.addBatch();

        if(i%1000 == 0) {
            ps.executeBatch();
            conn.commit(); // here
        }
    }
    ps.executeBatch();
    conn.commit(); // here

    System.out.println("Time Taken="+(System.currentTimeMillis()-start));

} catch (SQLException e) {
    e.printStackTrace();
}

Answer 2

您需要将autocommit设置为false

conn.setAutoCommit(false);

并在每个executeBath()之后手动执行提交

例