使用fputcsv（）函數將12,000條記錄從mysql表寫入CSV文件時出現內存和文件大小問題

Question

我有一個12,000條記錄的數據庫，我想將其寫入php代碼下面嘗試過的csv文件中，但是在文件大小變為980kb之后，數據開始刪除，此后文件大小開始減小。 已從mysql表中成功獲取數據。 CSV文件寫入問題。 請在下面查看我的代碼。

<?php
include './connection.php';

set_time_limit(0);
ini_set('memory_limit', '1024M');
$connection;
$db_link;

    $output_file = "db_data.csv";

    try {

        $csv_file = fopen($output_file,"b");
        $value = array('Name','Address','Phone Number','International Phone Number','Website');
            if(!empty($value)){
                fpassthru($csv_file);
                fputcsv($csv_file, $value,',');
                fclose($csv_file);
            }       
    } catch (Exception $e) {
        echo $message = $e->getMessage();
    }



        $connection = new Connection('1.2.3.8','admin','pass','automtn');
        $db_link = $connection->connect();
        $low = 0;
        $high = 100;
        $query = "SELECT name,formatted_address,formatted_phone_number,international_phone_number,website "
                . " FROM `fetch_data` ORDER BY id ASC LIMIT $low OFFSET $high";

        while($result = mysqli_query($db_link,$query)){
            $data = array();
            while ($row = mysqli_fetch_row($result)) {                
                $data[] = $row;                
            }
            write_to_csv($data);
            unset($data);
            $low+=100;
            $high+=100;
            $query = "SELECT name,formatted_address,formatted_phone_number,international_phone_number,website "
                . " FROM `fetch_data` ORDER BY id ASC LIMIT $low OFFSET $high";

        }

function write_to_csv($results) {

    global $output_file;
    $row = 0;
    try {

        $csv_file = fopen($output_file,"b");

        $fetched_data = $results;
        unset($results);
        foreach ($fetched_data as $value) {
            if(!empty($value)){
                fpassthru($csv_file);
                fputcsv($csv_file, $value, ',');
            }            
        }
        fclose($csv_file);
        unset($fetched_data);
    } catch (Exception $e) {
        echo $message = $e->getMessage();
    }
}


?>

Answer 1

您每次都寫入文件的開頭。 坦白說，我很驚訝模式“ b”本身不會引發錯誤； 除非文檔已過時。

使用附加模式寫“ a”（或“ ab”）

$csv_file = fopen($output_file,"ab");

---

由於您保留的是全局文件名，為什么不為自己省些麻煩，而是保留一個全局文件句柄。 您不必每次寫文件都花時間打開和關閉文件。

Answer 2

問題是我的高邏輯低，我用$ limit更改了它，它會保持不變，並且$ offset每次增加100，現在可以正常工作

<?php
include './connection.php';

set_time_limit(0);
ini_set('memory_limit', '1024M');
$connection;
$db_link;

    $output_file = "db_data.csv";

    try {

        $csv_file = fopen($output_file,"a+");
        $value = array('Name','Address','Phone Number','International Phone Number','Website');
            if(!empty($value)){
                fpassthru($csv_file);
                fputcsv($csv_file, $value,',');
                fclose($csv_file);
            }       
    } catch (Exception $e) {
        echo $message = $e->getMessage();
    }



        $connection = new Connection('1.2.3.8','admin','pass','automtn');
        $db_link = $connection->connect();
        $limit = 100;
        $offset = 0;

        $query = "SELECT name,formatted_address,formatted_phone_number,international_phone_number,website "
                . " FROM `fetch_data` ORDER BY id ASC LIMIT $limit OFFSET $offset ";

        while($result = mysqli_query($db_link,$query)){
            $data = array();
            while ($row = mysqli_fetch_row($result)) {                
                $data[] = $row;                
            }
            write_to_csv($data);
            unset($data);

            $offset +=100;
            $query = "SELECT name,formatted_address,formatted_phone_number,international_phone_number,website "
                . " FROM `fetch_data` ORDER BY id ASC LIMIT $low OFFSET $offset ";

        }

function write_to_csv($results) {

    global $output_file;
    $row = 0;
    try {

        $csv_file = fopen($output_file,"a+");

        $fetched_data = $results;
        unset($results);
        foreach ($fetched_data as $value) {
            if(!empty($value)){
                fpassthru($csv_file);
                fputcsv($csv_file, $value, ',');
            }            
        }
        fclose($csv_file);
        unset($fetched_data);
    } catch (Exception $e) {
        echo $message = $e->getMessage();
    }
}


?>