在MYSQL上合并数据的最佳方法

Question

I have a table that is populated twice a day with data collected from other tables, and I use it to create some reports.

The steps to achieve this is the following: Get all data that will be placed on the table, truncate the table and and then insert all data again.

Is it the best way to perform this in terms of performance? Isn't there a way to updating only things that really changed, insert the new data and skip the rest?

Answer 1

You can follow this for updating a table with merge:

UPDATE multiple tables in MySQL using LEFT JOIN

To manage the UPDATE or INSERT you can use this method:

Insert into a MySQL table or update if exists

Obviously all depends on your DB structure, what kind of data you are merging and, mainly, which keys are available.

Regards

Answer 2

CREATE TABLE new LIKE real;
INSERT INTO new ...;
RENAME TABLE real TO old, new TO real;
DROP TABLE old;

Pros/cons:

• You always have table real . (The RENAME is 'instantaneous' and 'atomic'.)
• It is simple.
• It may be slower than a combination of UPDATE and INSERT , but does that really matter, based on my first comment?

If there is more than you have let on, then see http://mysql.rjweb.org/doc.php/staging_table , which discusses other aspects of rapidly "ingesting" new data.