How can i merge two mysql databases of same characteristics with out losing any data

Question

Here is the issue. I have an app that was installed on two different servers with DB 1 and DB2. Each has the same schema and auto incremental key. but DB2 has more data input than DB1 while DB1 has some important data that is not in DB2.

How can I merge the two databases without losing my data in any of them?

I have searched and come across a guide that suggested this

INSERT INTO DB1.TABLE1 SELECT * FROM DB2.TABLE1

But my problem is that the databases have up to 106 tables.

is there a way to merge all the tables at once?

Answer 1

No, there is no easy button tool or procedure to merge rows from tables from two schemas.

We can envision the special case of a table that is guaranteed not to have any conflicting rows, where rows can be simply copied from one table to the other.

But in the more general case (a typical entity-relationship model implemented as tables in a relational database), we can't avoid data conflict issues due to UNIQUE KEY constraints, FOREIGN KEY constraints, et al.

For examle,

If table (id,name) in DB1 has row (3,'amigos') and in DB2 has row (3,'blind mice')

When we attempt to INSERT the row from DB2 into the table in DB1, and we get a unique key constraint violation on id=3, how do we handle that? Do we just not copy the row from DB2? Do we modify the id value on the row from DB2 to be (1113,'blind mice') when its copied into DB1?

How we handle these types of issues will depend. Is this a lookup table that is relatively static (a "domain" or "list of values" table), if the contents of the tables are the same in both databases, we would expect the contents of the merged table to be the same.

But we will handle that differently for a table of transactions where we expect the merged table to contain all the rows from DB1 and all the rows from DB2.

Q: "Is there a way to merge all the tables at once?"

A: No. If there was a tool do that kind of merge (if it did happen to do the right thing in a special case), with no doubt it will do the wrong thing on some of the 100+ tables.

Answer 2

This is all depends on how your auto incrementing keys are setup and how normalized/relational your data is. The more normalized your data is, the harder it will be. This isn't exactly an answer, but a general idea. This isn't an easy way, but probably the easiest .

1. Find the table with the largest auto incremented ID value in DB2. Lets pretend that you find that the Customer table's ID field has the most records and it's highest ID value is 91,365.
2. Find the table in DB1 that has the highest ID value. It's likely to be the same table, so again, let's pretend that it is Customer and it's highest ID is 7,104. Which specific table it is isn't important though.
3. Rounds up both values to give yourself some wiggle room: DB2: 100,000 DB1: 10,000
4. Write a query that disables auto increment on all DB2 tables, and reseeds each one of them to start incrementing at 120,000.
5. Write queries to import all records in DB1 tables, but add 100,000 to each DB1 key/foreignkey field. So record ids 1, 2, 15, 876, 1294 would become 100001, 100002, 100015, 100876, 101294.
6. Reenable auto incrementing. You should now have all of DB2s original records with ids below 100000, all of DB1s original records between 100000-120000, and all newly created records after the merge with IDs at 120000 and above.

This has the added advantage of giving you an easy way to segregate the old data from each database from the new data for trouble shooting in the future.

This is the easiest way to keep keys and foreign keys consistent without lots of lots of custom code.