PHP / MYSQL在数据库中存储大型数据集

Question

I am building a platform collecting and parsing "csv" files which contains 5000 rows average and 40 columns; files can be uploaded by any registered user so there should be no theoretically limit on how many files can be uploaded and stored (assuming I'm not having more than 20 uploads per day however). These file are being parsed by a custom made PHP parser which is kinda efficient but here comes the problem: storage. In particular, I would like to store these data in a MySQL db for later use: how should I organize my database? Should I create a table containing a row per file with each cell containing the content of each column of the original file? Or should I insert a single row for every row in the file? The first one seems to me kinda better because of the huge number of rows built up by the second solution but also less efficient because any time I have to extract any data in the database, I will have to parse again the whole blob then extract the data I need

Given the fact that i can't store the data in PHP arrays due to lack of memory if too many users connect at the same time, which would be the best solution in this case?

Answer 1

Since all the files have the same columns, you should store them in one table and add another an additional column (possibly referencing a second table with one row per upload) to uniquely identify a set of rows.

Eg:

rowId, setId, col1, col2, ...
1, 1, 'abc', 'def', ...
2, 1, 'abc', 'def', ...
3, 1, 'abc', 'def', ...
4, 2, 'abc', 'def', ...
5, 2, 'abc', 'def', ...
6, 2, 'abc', 'def', ...

This is what the data might look like for 2 sets of 3 rows each.