php从json在mysql中插入数据运行速度太慢

Question

我有以下代码来读取JSON并将结果存储在DDBB中。
该代码有效，但是只插入400条记录就花费了超过一分钟的时间。
如果我打开json，加载速度会很快。
我做错了什么？

    $db = new PDO('', '', '');
    $db->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);
    if(tableExists($db, 'locations') == 1)
    {
        $sql=$db->prepare("DROP TABLE locations");
        $sql->execute();
    }
    $sql ="CREATE TABLE `locations` (id INT(6) UNSIGNED AUTO_INCREMENT PRIMARY KEY, evid INT(6) NOT NULL, place VARCHAR(150), country VARCHAR(150), reg_date TIMESTAMP)" ;
    $db->exec($sql);
    $json = file_get_contents('thejson.php');
    $data = array();
    $data = json_decode($json); 
    foreach ($data as $key => $object) 
    {
        if(is_object($object))
        {
            $id = $object->id;
            $place = $object->name;
            $country = substr(strrchr($object->name, "-"), 2);
            $stmt = $db->prepare("INSERT INTO `locations` (evid, place, country) VALUES (:evid, :places, :country)");
            $stmt->bindValue(':evid', $id, PDO::PARAM_INT);
            $stmt->bindValue(':places', $place, PDO::PARAM_STR);  
            $stmt->bindValue(':country', $country, PDO::PARAM_STR);      
            $stmt->execute();
        }
    }

Answer 1

因此，我尝试的前两件事是将准备工作移出循环，并将其包装在事务中：

try {
    $db->beginTransaction();
    $stmt = $db->prepare("INSERT INTO `locations` (evid, place, country) VALUES (:evid, :places, :country)");

    foreach ($data as $key => $object) 
    {
        if(is_object($object))
        {
            $id = $object->id;
            $place = $object->name;
            $country = substr(strrchr($object->name, "-"), 2);

            $stmt->bindValue(':evid', $id, PDO::PARAM_INT);
            $stmt->bindValue(':places', $place, PDO::PARAM_STR);  
            $stmt->bindValue(':country', $country, PDO::PARAM_STR);      
            $stmt->execute();
        }
    }

    $db->commit();

} catch (Exception $e) {
    $db->rollBack();
    throw $e;
}

您可以做的另一件事是尝试使用bindParam通过引用绑定bindParam这样，您只需要在开始时对每个变量名称调用一次bindParam ，然后在每次迭代中覆盖这些变量的值并调用execute。

try {
    $db->beginTransaction();
    $stmt = $db->prepare("INSERT INTO `locations` (evid, place, country) VALUES (:evid, :places, :country)");
     $id = 0;
     $place = '';
     $country = '';

     $stmt->bindParam(':evid', $id, PDO::PARAM_INT);
     $stmt->bindParam(':places', $place, PDO::PARAM_STR);  
     $stmt->bindParam(':country', $country, PDO::PARAM_STR); 

    foreach ($data as $key => $object) 
    {
        if(is_object($object))
        {
            $id = $object->id;
            $place = $object->name;
            $country = substr(strrchr($object->name, "-"), 2);
            $stmt->execute();
        }
    }

    $db->commit();

} catch (Exception $e) {
    $db->rollBack();
    throw $e;
}

与此类似，而不是调用bind*您可以通过execute传递值：

try {
    $db->beginTransaction();
    $stmt = $db->prepare("INSERT INTO `locations` (evid, place, country) VALUES (:evid, :places, :country)");

    foreach ($data as $key => $object) 
    {
        if(is_object($object))
        {
            $params = array(
                ':id' => $object->id,
                ':places' => $object->name,
                ':country' => substr(strrchr($object->name, "-"), 2)
            );

            $stmt->execute($params);
        }
    }

    $db->commit();

} catch (Exception $e) {
    $db->rollBack();
    throw $e;
}

我怀疑使用事务会提高性能，但是我不知道切换绑定方法之间会有很多区别。

最好的选择是按照@PavanJiwnani的建议将所有记录插入单个查询中：

// first we need to compile a structure of only items 
// we will insert with the values properly transformed

$insertData = array_map(function ($object) {
     if (is_object($object)) {
        return array(
            $object->id,
            $object->name,
            substr(strrchr($object->name, "-"), 2)
        );
     } else {
       return false;
     }
}, $data);

// filter out the FALSE values
$insertData = array_filter($insertData);

// get the number of records we have to insert
$nbRecords = count($insertData);

// $records is an array containing a (?,?,?) 
// for each item we want to insert
$records = array_fill(0, $nbRecords, '(?,?,?)');

// now now use sprintf and implode to generate the SQL like:
// INSERT INTO `locations` (evid, place, country) VALUES (?,?,?),(?,?,?),(?,?,?),(?,?,?)
$sql = sprintf(
    'INSERT INTO `locations` (evid, place, country) VALUES %s',   
    implode(',', $records)
);

$stmt = $db->prepare($sql);

// Now we need to flatten our array of insert values as that is what 
// will be expected by execute()
$params = array();
foreach ($insertData as $datum) {
   $params = array_merge($params, $datum);
}

// and finally we attempt to execute
$stmt->execute($params);

Answer 2

尝试以毫秒为单位回显时间戳，以查看运行缓慢。 可能执行400个插入查询（包括打开/关闭连接）。

Answer 3

影响数据库性能的因素很多，请提供有关数据库系统，PHP版本和相关硬件的详细信息。

瓶颈可能在：

file_get_contents('thejson.php')

如果从远程主机获取JSON内容，即DB正常运行，则网络速度很慢。

您可能还需要考虑移动：

$stmt = $db->prepare("INSERT INTO `locations` (evid, place, country) VALUES (:evid, :places, :country)");

走出foreach循环。