Importing data from files to database by the users of the site

Question

It's necessary to allow users to import their data on the site (in mysql database). Every user who logs in, can import data from a file to the database and then work with them on the site. It takes a long time to import the data and creates a high load on the database as well. Could you please tell how to queue importing better? So the data would be imported at one online, but to queue data to be written periodically to the database. Thank you.

Answer 1

Perhaps insert delayed would help. From the manual:

Another major benefit of using INSERT DELAYED is that inserts from many clients are bundled together and written in one block. This is much faster than performing many separate inserts.

Answer 2

My suggestion would be to upload the .CSV file and post a message into a queue (or post the message on some other event like click on "Import" button). This way you will be able to return immediately and display a message like "Please wait for import". Then once the import is done and the database is updated the HTML page can refresh itself and display proper status. Depending on the servers and/or database load you can have only one or multiple simultaneous imports going on at the same time.