Best way to transfer data between SQL Servers

Question

I have a general in-office database and a bunch of laptop-installed SQL Server Express databases of the same structure. In these databases I log information, which is partitioned by the id of a specific event. The laptops are taken by employees mostly to service one event (2-3 at the most sometimes), and obviously I have an eventId column in all tables except the dictionaries in the databases (though it rarely goes above 3 on the laptop databases).

When the employees come back to the office, they need to backup the last event data to the office SQL Server, where eventId 's can be a hundred or a thousand, etc., this is the history of all past events.

Another reason for my question is I would like to be able to make an eventId -based selective copy of an existing database to a different server for development purposes (from local to SQL Azure actually).

What is the efficient way of moving such data between servers? I have approximately 50-100 tables with keys and references.

1. What we do at the moment - selective copying the data to a new clean database on source server (only whatever is required for copying); backing up this new DB; restoring it on a target server; data copy between databases using dynamic T-SQL with a dbname as a parameter. What I don't like is dynamic T-Sql, no syntax check at all until runtime.

2. Loading data somewhat in RAM with help of developing a C# program (but it may be too resource consuming, as long as data may be large) and transferring it to a new server with a C# program as a proxy and RAM as intermediate storage. Challenging and in certain situations impossible (too much data).

3. I started looking into SSIS, but it seems that the general advice for use of SSIS is when copying is done at fixed environment ("in-house"). We supply software not to a vast amount of customers, but there surely are more than one, and we would need to pass connection information dynamically from somewhere, as long as we can't visit all our customers to prepare a set-up for them on location.

4. Any other suggestions?

Any help appreciated!

Answer 1

Here is what I would recommend.

Create an SSIS package. Install it on the Laptops SQL Server Express.

You can use a configuration file that will allow you to select which Event Ids you want to Export.

You can also keep all of your connection information in the configuration file. For those non in-house customers you can remove the configuration file between exports.

That would be my recommendation.