如何与数据库双向同步列表？

Question

I'm building a system that needs to keep two external sources in sync with each other and keep a history of these syncs. Here's a quick diagram I made to explain my problem:

If source 1 changes (add, remove, both), when my system runs the periodic sync, I want source 2 to be identical to source 1. The same goes for any changes to source 2. Bearing in my mind that both source 1 & 2 can change independantly between syncs.

The solution I have come up with so far is to have a central database that saves the content of source 1 & source 2 on first sync minus any duplicates and then replace the content of source 1 & 2 with the content of the database.

Then on subsequent syncs I would compare source 1 & 2 with the current database and save the new state in a new database. This way I would keep a history of the state of the database after each sync.

So my two main concerns at this point are:

1. Would my solution work and is it a "good" way of achieving what I want?
2. How would I design this system (schema) and keeping a track record of syncs in database?
3. Currently there are two external sources to sync, however I plan to be able to add external sources that need to be synced too (maybe 3 or 4). Is my current solution scaleable or am I missing something?

I'm mainly proficient in Java and JS if code/pseudo-code is easier to explain. Any help is greatly apreaciated, thanks.

Answer 1

I think you would need a distributed cache/in memory data store like Redis/Hazelcast which would make sure both your sources are aware of the changes being made. You can just add a listener to this cache and persist it in the database, this would make your data consistent.