Fastest way to replace data in a table from a temporary table in MySQL

Question

I have a need to "update" some table data I receive from external source (every time I receive "all" data, with some fields for some records updated). There's no unique field or combination of fields, and thus I figured the best way would be to every time to wipe out all data from DB and write all (now updated) data in again. There are up to a 1000 records (there will never be more than that), about 15 short fields each: text, numbers, datetime. And I'm writing it to remote DB (so, it's slow).

Currently I'm doing:

delete from `table` where `date_dt` > ?

and then for each row

INSERT INTO `table` ( `field_0`,`field_1`,... ) VALUES (?,?,...)

It's not only slow, but it's possible that the end user may not see the complete data while I'm still inserting.

I figured I could do:

CREATE TEMPORARY TABLE `temp_table` ( ... ); -- same structure as in main table
INSERT INTO `temp_table` ( `field_0`,`field_1`,... ) VALUES (?,?,...)   -- repeat 1000x

START TRANSACTION;
DELETE FROM `table`;
INSERT INTO `table` SELECT * FROM `temp_table`;
DROP `temp_table`;
COMMIT;

Does this makes any sense? What's is a better way of solving this?

The speed of filling up the temp table with data is not crucial, but filling the main table with data is (so users don't see incomplete data, or the period of time they do is minimal).

Answer 1

mysqlimport --delete will truncate the table first, and then load your external data from a CSV file. It runs many times faster than doing INSERT one row at a time.

See https://dev.mysql.com/doc/refman/5.7/en/mysqlimport.html

I did a presentation in April 2017 about performance of bulk data loads for MySQL: https://www.slideshare.net/billkarwin/load-data-fast

PS: Don't use the temp table solution if you have a MySQL replication environment. This is a well-known way of breaking replication. If the slave restarts in between your creation of the temp table and the INSERT...SELECT that reads from the temp table, then the slave will find the temp table is gone, and this will result in an error and stop replication. This might seem unlikely, but it does happen eventually.