增量MySQL

Question

I have 2 MySQL databases on the same Linux box. They aren't super large, but some tables hold around 500,000 rows, increasing by about 40,000 rows per month.

What I need to do is to write a partial backup from one database to the other once per day. This partial backup is a snapshot and apart from the backups will not have any fresh data written to it. It contains only some of the tables of the main db, and from those tables only some of the fields.

It is easy enough to write a PHP script that deletes the backup database, and then recreates it with the desired data to get a fresh snapshot, however i am wondering if there is a way to do this incrementally with PHP and only write new or changed data.

Answer 1

Do you have a timestamp field on your database? Using a timestamp with ON UPDATE CURRENT_TIMESTAMP clause would allow you to know the modification time of each row. That way, you could easily do a SELECT query on rows WHERE timestamp is greater than a given value.

Answer 2

If you are checking for new or changed data on that many records, it will be heavy on the machine. Each record will have to be compared to a record in the other database. Will be very slow and huge performance impact.

The best way to go would be to use software for backup that is designed for this task. If you realy want a PHP solution I would add a new table to the database called changes. It will hold a table name and key index.

Then add a trigger on each table that needs a backup that will enter a new entry in the changes table with the table name and the key. Then your PHP script can query this table to find changes and new items and only query these.

Instead of using a seperate table you can always add a tiny int column to the existing tables called lastAction or something and store 0 for no change, 1 for update and 2 for new record.

Then again you can query only the needed data.

Keep in mind that a PHP solution with two database is never reliable. You would need a structure like:

1 Query record from old database
2 Update/insert record in new database
3 Mark old database record as completed.

It could very well be that after you are done with step 2, that you lose connection to the old database and can no longer set it to completed. This will result in the record being updated or added again on the next run.

Another issue could be that between step 1 and 2, somebody alters the record again. You then insert an older value in the backup and mark it as completed. While the newest update will no longer be backed up.