简体   繁体   中英

PDO transaction timing out when it has 10 insert (to mysql) statements

I'm fairly new to PDO - trying to optimize some code that previously updated a MySQL table one row at a time, and get it to instead do the update in a batch, because there are literally thousands of rows.

Long story short, the code needs to get a recursive directory listing, and based on that, update the table of file urls, indicating whether the table is found. That's the end goal... here are the basic steps that the code is performing:

  1. clear the "directory" table
  2. do a recursive scandir into a temporary array, thousands of items long
  3. go through the array one by one, and insert into the "directory" table
  4. run an update query on the main table setting filefoundstatus based on the content of the "directory" table

Step 3 is where I'm having a problem. I keep getting "PHP Fatal error: Maximum execution time of 90 seconds exceeded." I've decreased the transaction size to 10, and increased the max_execution_time to 90, and it does manage to insert 1720 rows before it crashes with the above error.

Here's the code for step 3 (including a bit of step 2) - $pdo is a PDO object. If anybody has any pointers, it would be much appreciated. :)

//get the dir list into a one-dimensional array
$dirlistall = scanAllDir($rootdirprefix . 'docs/sds');
$countDirsList = count($dirlistall);
//print_r($dirlistall);

//load the dir list into the Directory Listings table
$stmt = $pdo->prepare("INSERT INTO DirectoryListingResults (URL) VALUES (:url)");
$batchSize = 10;
$i = 0;
try {
    for ($idx=0; $idx*$batchSize < $countDirsList; $idx++) { 
        $dirsPartial = array_slice($dirlistall, $idx*$batchSize, $batchSize);

        $pdo->beginTransaction();
        $stmt->bindParam(':url', $url);
        foreach ($dirsPartial as $url)
        {
            $stmt->execute();
        }
        $pdo->commit();

        unset($dirsPartial);
    }
}catch (Exception $e){
    $pdo->rollback();
    throw $e->getMessage();
}
unset($dirlistall);

Running the code on my test machine, which is running Windows 10 with IIS, and PHP 5.6. The MySQL server is remote, version 5.6.43.

[Edit]: In case it's relevant, here are the PDO connection options:

$pdooptions = [
    PDO::ATTR_ERRMODE            => PDO::ERRMODE_EXCEPTION,
    PDO::ATTR_DEFAULT_FETCH_MODE => PDO::FETCH_ASSOC,
    PDO::ATTR_EMULATE_PREPARES   => false,
];

I'm curious if binding values instead of parameters could help, so instead of this

$stmt->bindParam(':url', $url);
foreach ($dirsPartial as $url)
{
    $stmt->execute();
}

this

foreach ($dirsPartial as $url)
{
    $stmt->execute([':url' => $url]);
}

I really don't know if that will help the performance, but I know since parameters are bound by reference that the redefinition of $url in the foreach loop could cause problems in general.

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM