php從json在mysql中插入數據運行速度太慢

Question

我有以下代碼來讀取JSON並將結果存儲在DDBB中。
該代碼有效，但是只插入400條記錄就花費了超過一分鍾的時間。
如果我打開json，加載速度會很快。
我做錯了什么？

    $db = new PDO('', '', '');
    $db->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);
    if(tableExists($db, 'locations') == 1)
    {
        $sql=$db->prepare("DROP TABLE locations");
        $sql->execute();
    }
    $sql ="CREATE TABLE `locations` (id INT(6) UNSIGNED AUTO_INCREMENT PRIMARY KEY, evid INT(6) NOT NULL, place VARCHAR(150), country VARCHAR(150), reg_date TIMESTAMP)" ;
    $db->exec($sql);
    $json = file_get_contents('thejson.php');
    $data = array();
    $data = json_decode($json); 
    foreach ($data as $key => $object) 
    {
        if(is_object($object))
        {
            $id = $object->id;
            $place = $object->name;
            $country = substr(strrchr($object->name, "-"), 2);
            $stmt = $db->prepare("INSERT INTO `locations` (evid, place, country) VALUES (:evid, :places, :country)");
            $stmt->bindValue(':evid', $id, PDO::PARAM_INT);
            $stmt->bindValue(':places', $place, PDO::PARAM_STR);  
            $stmt->bindValue(':country', $country, PDO::PARAM_STR);      
            $stmt->execute();
        }
    }

Answer 1

因此，我嘗試的前兩件事是將准備工作移出循環，並將其包裝在事務中：

try {
    $db->beginTransaction();
    $stmt = $db->prepare("INSERT INTO `locations` (evid, place, country) VALUES (:evid, :places, :country)");

    foreach ($data as $key => $object) 
    {
        if(is_object($object))
        {
            $id = $object->id;
            $place = $object->name;
            $country = substr(strrchr($object->name, "-"), 2);

            $stmt->bindValue(':evid', $id, PDO::PARAM_INT);
            $stmt->bindValue(':places', $place, PDO::PARAM_STR);  
            $stmt->bindValue(':country', $country, PDO::PARAM_STR);      
            $stmt->execute();
        }
    }

    $db->commit();

} catch (Exception $e) {
    $db->rollBack();
    throw $e;
}

您可以做的另一件事是嘗試使用bindParam通過引用綁定bindParam這樣，您只需要在開始時對每個變量名稱調用一次bindParam ，然后在每次迭代中覆蓋這些變量的值並調用execute。

try {
    $db->beginTransaction();
    $stmt = $db->prepare("INSERT INTO `locations` (evid, place, country) VALUES (:evid, :places, :country)");
     $id = 0;
     $place = '';
     $country = '';

     $stmt->bindParam(':evid', $id, PDO::PARAM_INT);
     $stmt->bindParam(':places', $place, PDO::PARAM_STR);  
     $stmt->bindParam(':country', $country, PDO::PARAM_STR); 

    foreach ($data as $key => $object) 
    {
        if(is_object($object))
        {
            $id = $object->id;
            $place = $object->name;
            $country = substr(strrchr($object->name, "-"), 2);
            $stmt->execute();
        }
    }

    $db->commit();

} catch (Exception $e) {
    $db->rollBack();
    throw $e;
}

與此類似，而不是調用bind*您可以通過execute傳遞值：

try {
    $db->beginTransaction();
    $stmt = $db->prepare("INSERT INTO `locations` (evid, place, country) VALUES (:evid, :places, :country)");

    foreach ($data as $key => $object) 
    {
        if(is_object($object))
        {
            $params = array(
                ':id' => $object->id,
                ':places' => $object->name,
                ':country' => substr(strrchr($object->name, "-"), 2)
            );

            $stmt->execute($params);
        }
    }

    $db->commit();

} catch (Exception $e) {
    $db->rollBack();
    throw $e;
}

我懷疑使用事務會提高性能，但是我不知道切換綁定方法之間會有很多區別。

最好的選擇是按照@PavanJiwnani的建議將所有記錄插入單個查詢中：

// first we need to compile a structure of only items 
// we will insert with the values properly transformed

$insertData = array_map(function ($object) {
     if (is_object($object)) {
        return array(
            $object->id,
            $object->name,
            substr(strrchr($object->name, "-"), 2)
        );
     } else {
       return false;
     }
}, $data);

// filter out the FALSE values
$insertData = array_filter($insertData);

// get the number of records we have to insert
$nbRecords = count($insertData);

// $records is an array containing a (?,?,?) 
// for each item we want to insert
$records = array_fill(0, $nbRecords, '(?,?,?)');

// now now use sprintf and implode to generate the SQL like:
// INSERT INTO `locations` (evid, place, country) VALUES (?,?,?),(?,?,?),(?,?,?),(?,?,?)
$sql = sprintf(
    'INSERT INTO `locations` (evid, place, country) VALUES %s',   
    implode(',', $records)
);

$stmt = $db->prepare($sql);

// Now we need to flatten our array of insert values as that is what 
// will be expected by execute()
$params = array();
foreach ($insertData as $datum) {
   $params = array_merge($params, $datum);
}

// and finally we attempt to execute
$stmt->execute($params);

Answer 2

嘗試以毫秒為單位回顯時間戳，以查看運行緩慢。 可能執行400個插入查詢（包括打開/關閉連接）。

Answer 3

影響數據庫性能的因素很多，請提供有關數據庫系統，PHP版本和相關硬件的詳細信息。

瓶頸可能在：

file_get_contents('thejson.php')

如果從遠程主機獲取JSON內容，即DB正常運行，則網絡速度很慢。

您可能還需要考慮移動：

$stmt = $db->prepare("INSERT INTO `locations` (evid, place, country) VALUES (:evid, :places, :country)");

走出foreach循環。