How to copy data from one table to another in SQL Server

Question

I am trying to copy data from views on a trusted SQL Server 2012 to tables on a local instance of SQL Server on a scheduled transfer . What would be the best practice for this situation?

Here are the options I have come up with so far:

1. Write an executable program in C# or VB to delete existing local table, query the data from remote database and then write results to tables in the local database. The executable would run on a scheduled task.

2. Use BCP to copy data to a file and then upload into local table.

3. Use SSIS

Note: The connection between local and remote SQL Server is very slow.

Answer 1

Since the transfers are scheduled, so I suppose you want this data to be up-to-date.

My recommendation would be to use SSIS and schedule it using SQL Agent. If you wrote a C# program, I think the best outcome you will gain is a program imitating SSIS. Moreover, SSIS will be a very easy to amend the workflow anytime.

Either way, to make such program/package up-to-date, you will have to answer an important question: Is the source table updatable or is it like a log (inserts only)? This question is so important because it will determine how you will fetch the new updates from the source table. For example, if the table represents logs, you will most probably use the Primary Key to detect new records, if not, you might want to seek a column representing update date/time. If you have the authority to alter the source table, you might want to add timestamp column which represent the row version ( timestamp differs than datetime )

For building an SSIS package, it will mainly contain the following components:

1. Execute SQL Task to get the maximum value from source table.

2. Execute SQL Task to get the last value where it should start from at the destination table. You can get this value either by selecting the maximum value from the destination table or if the table is pretty large you can store that value in another table (configuration table for example).

3. Data Flow which moves the data from source table starting after the value fetched in step 2 to the value fetched in step 1.

4. Execute SQL Task for updating the new maximum value back to the configuration table if you chose this technique.

Answer 2

BCP can be used to export the data compress and transfer over network which can be then imported into local instance of SQL. Also with BCP data exports can be contained with smaller batches of data for easier management of data.

• https://msdn.microsoft.com/en-us/library/ms191232.aspx
• https://technet.microsoft.com/en-us/library/ms190923(v=sql.105).aspx