如何通过PHP将大型CSV文件导入MySQL？

Question

I've followed multiple tutorials on how to get this done and I get the message "Imported Successfully" or similar message depending on the script I've used, but I have not had any luck doing so. 我已经按照有关如何完成此操作的多个教程进行了学习，但根据我使用的脚本，我会收到消息“已成功导入”或类似的消息，但是我没有运气。 My CSV consists of 26 columns and about 18,000 rows. 我的CSV包含26列和大约18,000行。 I've set up the PHPMYADMIN database and table and I have tried the same script with a small load of data consisting of only 4 columns, and that one worked, I've attempted to figure out how to get the others to work, but have had 0 luck. 我已经建立了PHPMYADMIN数据库和表，并尝试了相同的脚本，只加载了少量数据，仅包含4列，而且其中一个起作用，我试图弄清楚如何使其他脚本起作用，但是运气是0。

<?php
$conn = mysqli_connect("localhost", "DB_USERNAME", "DB_PASSWORD", "DB_NAME");

if (isset($_POST["import"])) {

    $fileName = $_FILES["file"]["tmp_name"];

    if ($_FILES["file"]["size"] > 0) {

        $file = fopen($fileName, "r");

        while (($column = fgetcsv($file, 10000, ",")) !== FALSE) {
            $sqlInsert = "INSERT into TABLE_NAME (COL1,COL2,COL3,COL4,COL5,COL6,COL7,COL8,COL9,COL10,COL11,COL12,COL13,COL14,COL15,COL16,COL17,COL18,COL19,COL20,COL21,COL22,COL23,COL24,COL25,COL26)
                   values ('" . $column[0] . "','" . $column[1] . "','" . $column[2] . "','" . $column[3] . "','" . $column[4] . "','" . $column[5] . "','" . $column[6] . "','" . $column[7] . "','" . $column[8] . "','" . $column[9] . "','" . $column[10] . "','" . $column[11] . "','" . $column[12] . "','" . $column[13] . "','" . $column[14] . "','" . $column[15] . "','" . $column[16] . "','" . $column[17] . "','" . $column[18] . "','" . $column[19] . "','" . $column[20] . "','" . $column[21] . "','" . $column[22] . "','" . $column[23] . "','" . $column[24] . "','" . $column[25] . "')";
            $result = mysqli_query($conn, $sqlInsert);

            if (! empty($result)) {
                $type = "success";
                $message = "CSV Data Imported into the Database";
            } else {
                $type = "error";
                $message = "Problem in Importing CSV Data";
            }
        }
    }
}
?>
<!DOCTYPE html>
<html>

<head>
<script src="jquery-3.2.1.min.js"></script>

<style>
body {
    font-family: Arial;
    width: 550px;
}

.outer-scontainer {
    background: #F0F0F0;
    border: #e0dfdf 1px solid;
    padding: 20px;
    border-radius: 2px;
}

.input-row {
    margin-top: 0px;
    margin-bottom: 20px;
}

.btn-submit {
    background: #333;
    border: #1d1d1d 1px solid;
    color: #f0f0f0;
    font-size: 0.9em;
    width: 100px;
    border-radius: 2px;
    cursor: pointer;
}

.outer-scontainer table {
    border-collapse: collapse;
    width: 100%;
}

.outer-scontainer th {
    border: 1px solid #dddddd;
    padding: 8px;
    text-align: left;
}

.outer-scontainer td {
    border: 1px solid #dddddd;
    padding: 8px;
    text-align: left;
}

#response {
    padding: 10px;
    margin-bottom: 10px;
    border-radius: 2px;
    display:none;
}

.success {
    background: #c7efd9;
    border: #bbe2cd 1px solid;
}

.error {
    background: #fbcfcf;
    border: #f3c6c7 1px solid;
}

div#response.display-block {
    display: block;
}
</style>
<script type="text/javascript">
$(document).ready(function() {
    $("#frmCSVImport").on("submit", function () {

        $("#response").attr("class", "");
        $("#response").html("");
        var fileType = ".csv";
        var regex = new RegExp("([a-zA-Z0-9\s_\\.\-:])+(" + fileType + ")$");
        if (!regex.test($("#file").val().toLowerCase())) {
                $("#response").addClass("error");
                $("#response").addClass("display-block");
            $("#response").html("Invalid File. Upload : <b>" + fileType + "</b> Files.");
            return false;
        }
        return true;
    });
});
</script>
</head>

<body>
    <h2>Import CSV file into Mysql using PHP</h2>

    <div id="response" class="<?php if(!empty($type)) { echo $type . " display-block"; } ?>"><?php if(!empty($message)) { echo $message; } ?></div>
    <div class="outer-scontainer">
        <div class="row">

            <form class="form-horizontal" action="" method="post"
                name="frmCSVImport" id="frmCSVImport" enctype="multipart/form-data">
                <div class="input-row">
                    <label class="col-md-4 control-label">Choose CSV
                        File</label> <input type="file" name="file"
                        id="file" accept=".csv">
                    <button type="submit" id="submit" name="import"
                        class="btn-submit">Import</button>
                    <br />

                </div>

            </form>

        </div>
               <?php
            $sqlSelect = "SELECT * FROM TABLE_NAME";
            $result = mysqli_query($conn, $sqlSelect);

            if (mysqli_num_rows($result) > 0) {
                ?>
            <table id='userTable'>
<?php

                while ($row = mysqli_fetch_array($result)) {
                    ?>

                <tbody>
                <tr>
                    <td><?php  echo $row['id']; ?></td>
                    <td><?php  echo $row['firstname']; ?></td>
                    <td><?php  echo $row['lastname']; ?></td>
                    <td><?php  echo $row['number']; ?></td>
                </tr>
                    <?php
                }
                ?>
                </tbody>
        </table>
        <?php } ?>
    </div>

</body>

</html>

Answer 1

What you've got is hugely inefficient and very insecure. 您所拥有的效率非常低下并且非常不安全。 As soon as you have an apostrophe in your data the whole thing is going to explode, for example. 例如，一旦您的数据中有撇号，整个事情就会爆发。

Prepared statements protect against data injection , and also greatly reduce the overhead needed to perform repeated queries. 准备好的语句可以防止数据注入，还可以大大减少执行重复查询所需的开销。 This is possible with mysqli, but PDO is much less verbose and more modern – its use should be preferred in new code. 对于mysqli，这是可能的，但是PDO不太冗长，更现代-在新代码中应首选使用PDO。

Something like this should be a good start, but you'll want to do some proper error checking using try/catch. 这样的事情应该是一个好的开始，但是您将需要使用try / catch进行一些正确的错误检查。 It assumes that the number of columns in the CSV is equal to the number of columns you want to insert into the database. 假定CSV中的列数等于要插入数据库中的列数。

$conn = new \PDO("mysql:host=localhost;dbname=DB_NAME", "DB_USERNAME", "DB_PASSWORD");

if (isset($_POST["import"])) {

    $fileName = $_FILES["file"]["tmp_name"];
    if ($_FILES["file"]["size"] > 0) {
        $file = fopen($fileName, "r");

        $sql = "INSERT INTO TABLE_NAME (COL1, COL2, COL3, COL4, COL5, COL6, COL7, COL8, COL9, COL10, COL11, COL12, COL13, COL14, COL15, COL16, COL17, COL18,COL19, COL20, COL21, COL22, COL23, COL24, COL25, COL26)
                VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)"
        $stmt = $conn->prepare($sql);
        while (($column = fgetcsv($file, 10000, ",")) !== FALSE) {
            $result = $stmt->execute($column);
        }
    }
}

如何通过PHP将大型CSV文件导入MySQL？

问题描述

1 个解决方案

解决方案1
1 2018-10-24 18:15:27

如何通过PHP将大型CSV文件导入MySQL？

问题描述

1 个解决方案

解决方案1 1 2018-10-24 18:15:27

解决方案1
1 2018-10-24 18:15:27