SQL data transfer from local to live server

Question

I am working on a university project. What I need is to write a C# program that periodically updates the server based on changes in local. Basically the program is for a large geographical data set (60 Gb), and the tables are simple with no foreign keys. Local is updated almost daily, and currently the changes are updated in server once or twice a month.

I need some ideas about how to do this efficiently and optimally. I do not want to overwrite the previous records on my live server just update them and keep a record of changes and any deletions. I am using the Express version of SQL server 2008 R2 and VS Express 2012.

Answer 1

Add 2 columns to your table (both locally and on server): time_updated and is_deleted .

Change your software to update time_updated timestamp any time record is updated, and also change is_deleted to reflect that record is to be deleted (rather than actually deleting it). All user-visible select queries should filter by is_deleted to hide "deleted" rows from view.

Create index on time_updated , and when you need to sync data to remote server, do so only by timestamp since last update.

Doing this should allow you to only sync updates and deletions that happened since last sync.