通过PHP向MySQL数据库添加100,000多条记录的最佳方法是什么？

Question

I'm developing an application and have over 100,000 records, that have been read in from a file, which need to be inserted into a MySQL database.

What would be the best approach of inserting these to the database?

The approach I'm currently working from is generating a SQL query to insert all the records in one call to the database.

INSERT INTO table (field1, field2)
VALUES
(123, 456),
(125, 984),
...

Is there a limit to the number of records that can be inserted at once? And in terms of performance is this the best method?

I have considered an alternative of splitting the records into a number of queries but I'm unsure if this would have any benefit?

Any advice would be greatly appreciated, thanks!

Answer 1

The way you're doing it is indeed very efficient. The maximum query length is determined by the max_allowed_packet setting so you may need to split the query into several queries, see http://dev.mysql.com/doc/refman/5.5/en/server-system-variables.html#sysvar_max_allowed_packet

Answer 2

I would recommend adding one row at a time with LOW_PRIORITY or DELAYED depending on your engine. The reason for this is that other users of your table will still have access to read and write once in a while instead of having the entire table locked for an extended period of time.

Answer 3

Permissions and/or format not withstanding, using

load data infile to import the file directly

Answer 4

LOAD DATA INFILE is probably the best way to go. Especially if the data file you're reading from is already in a format that can be used by it.

Answer 5

您也可以考虑使用MySQL的LOAD DATA INFILE命令从CSV文件中插入数据。