PHP PDO Mysql Insert Performance

Question

I normally don't use a lot of SQL stuff in PHP but recently have been forced into it by a friend as to help him debug something.

I am using PDO with PHP to insert some data but the insert performance is abysmal. A simple loop of 151 inserts takes nearly 6 seconds! and I am lost as to why.

Here is my code:

<?php

$database='XXXXXX';
$username='XXXXXX';
$password='XXXXXX';
$hostname='127.0.0.1';

$inserted=0;
$counted=0;
$dsn = 'mysql:dbname='.$database.';host='.$hostname.'';
$start=microtime(true);
try {
    $data = new PDO($dsn, $username, $password, array(PDO::ATTR_EMULATE_PREPARES => false));
} catch (PDOException $e) {
    echo('Connection failed: ' . $e->getMessage());
}
for($i=1;$i<=150;$i++) {
    $time=time();
    $query=$data->prepare("INSERT INTO `tbl_temp` (aid, bid) VALUES (?, ?)");
    $query->execute(array($i, $time));
}
$data=null;
print "Took: ".(microtime(true)-$start)." S to insert $i records\n";

// Took: 5.569482088089 S to insert 151 records <--- result
?>

I have tried the same code using bindParam also and the speed is roughly the same. The server has an 8 core Xeon processor and 64gb of RAM. The script is being run from the command line (php-cgi) and the database and table are new and empty. The database type is InnoDB. Can anyone point me in the right direction of where to look as to why it's so slow? Because I am sure MySQL never used to be this slow!

Answer 1

found this one. Performance in PDO / PHP / MySQL: transaction versus direct execution . so try this one.

$query = $data->prepare("INSERT INTO `tbl_temp` (aid, bid) VALUES (?, ?)");
try {
        $data->beginTransaction();
        for($i=0; $i < 150; $i++) {
            $time = time();
            $query->bindValue(1, $i, PDO::PARAM_INT);
            $query->bindValue(2, $time, PDO::PARAM_STR);
            $query->execute();
        }
        $data->commit();
    } catch(PDOException $e) {
            $data->rollBack();
    }

Answer 2

Turns out that a setting inside the SQL server is what was causing it. As far as I can tell the transaction commit was set to flush to disk on every write:

innodb_flush_log_at_trx_commit=0

This is the default ACID compliant settings of a new install.

I changed my setting to this

innodb_flush_log_at_trx_commit=2

This allows ONLY power outage or OS crash to erase the transaction buffer/log and not a mysqld crash.

For some people who need the "D" of ACID to be 100% true then you should leave this setting alone.

Answer 3

As Simon stated in a comment, there is no use of doing the preparation within the loop. Doing so will cause 300 queries to be sent to the database, each time one for the preparation and on for the actual insert. Using the prepare-statement before will cause only 151 queries:

$query = $data->prepare("INSERT INTO `tbl_temp` (aid, bid) VALUES (?, ?)");
for ($i = 1; $i <= 150; $i++) {
    $time = time();
    $query->execute(array($i, $time));
}

Another idea could be to use instead a combined multi insert statement. I guess it could have a better performance, but I'm not quite sure:

$query = 'INSERT INTO `tbl_temp` (aid, bid) VALUES';
for ($i = 1; $i <= 150; $i++) {
    if ($i == 1) {
        $query .= ' ('.$i.', '.time().')';
    } else {
        $query .= ', ('.$i.', '.time().')';
    }
}
$data->exec($query);