简体繁体 English

将数据/ SQL负载写入文本文件的最安全方法是什么？

[英]What's the safest way to write loads of data / SQL to a text file?

原文 2013-01-22 12:16:23 6 1 php/ mysql/ sql

I'm facing an issue with query / data integrity. 我面临查询/数据完整性问题。

I'm reading loads of data (many gigabytes) from a MySQL database, processing them using PHP, to re-insert them into the same database in completely different tables. 我正在从MySQL数据库读取大量数据（许多GB），使用PHP处理它们，以将它们重新插入到完全不同的表中的同一数据库中。 It's a one-run conversion from an old data structure to a new one, where tons of values need on the fly modification. 这是从旧数据结构到新结构的一次性运行，其中需要即时修改大量的值。

SELECT FROM x -> modify in PHP -> INSERT INTO y is way too slow. SELECT FROM x - > modify in PHP - > INSERT INTO y是太慢。 That takes around 1 second per 1000 records, and we're talking many many millions here. 每1000条记录大约需要1秒钟的时间，而我们在这里谈论的数以百万计。 That's why I decided to export every INSERT into a text/sql file. 这就是为什么我决定将每个INSERT导出到text / sql文件中的原因。

But, what's the safest way to go here? 但是，去这里最安全的方法是什么？ The data is everything from booleans to BLOB's. 数据是从布尔值到BLOB的所有数据。 I can't really be sure of what's in it and what's not, but everything needs to get back into place appropriately, including quotes, slashes, etc. 我不能真正确定其中有什么，没有什么，但是一切都需要适当地恢复原状，包括引号，斜杠等。

How can I escape without any data loss? 如何逃脱而不会丢失任何数据？

1 个解决方案

You can use prepared statements to avoid the need to escape the data. 您可以使用准备好的语句来避免转义数据的需要。

The queries you create for the prepared statements can be concatenated and executed as a group, instead of having one execute call for each row. 您为准备好的语句创建的查询可以串联并作为一个组执行，而不是对每一行执行一次执行调用。

The code that creates, groups & concatenates SQL statements together should allow you to easily choose the number of statements concatenated in a group so you can experiment with 100 statements in one exec call vs. 1000 statements in one exec call. 将SQL语句创建，分组和连接在一起的代码应使您能够轻松地选择一组中连接的语句数，因此您可以在一个exec调用中试验100个语句，而在一个exec调用中试验1000个语句。

The results should be much better. 结果应该好得多。