如何从PHP服务器生成和导出大CSV文件？

Question

My server can be PHP or nodeJS, prefer staying in PHP but if it can't be done/better in node i'll appreciate the answer.

I want to generate and export a very big CSV file, the problem is that I can't load all the data from MySQL once, so currently I limit my data to some amount which doesn't crash the app, but it's slow and most data won't be exported.

I thought of this solutions to generate and export the CSV file:

1. Send 1..N calls to my server, each call will generate Part1...PartN CSV and the browser will append them [won't the browser crash?]
2. In one request, stream the data to the browser, but then how does the browser start downloading the file?

I don't mind the client will wait a lot, just want a good way to export ~200MB csv from the data I got in MySQL.

Thanks.

Answer 1

This is a nice solution using Mysql notation:

SELECT *
INTO OUTFILE '/tmp/dbexport.csv' 
FIELDS ENCLOSED BY '"' 
TERMINATED BY ';' 
ESCAPED BY '"' 
LINES TERMINATED BY '\r\n'
FROM tables;

Have a look at this answer too.

You could consider also use a .tsb (tab separated file) it does make no difference for you:

mysql your_database -e "select * from tables" -B > export.tsv