从 On Prem 到 Azure SQL (PaaS) 的数据迁移

Question

We have an on-prem SQL Server DB (SQL Server 2017 Comp 140) that is about 1.2 TB. We need to do a repeatable migration of just the data to an on cloud SQL (Paas). The on-prem has procedures and functions that do cross DB queries which eliminates the Data Migration Assistant. Many of the tables that we need to migrate are system versioned tables (just to make this more fun). Ideally we would like to move the data into a different schema of a different DB so we can avoid the use of External tables (worried about performance).

Moving the data is just the first step as we also need to do an ETL job on the data to massage it into the new table structure.

We are looking at using ADF but it has trouble with versioned tables unless we turn them off first.

What are other options that we can look and try to be able to do this quickly and repeatedly? Do we need to change to IaaS or use a third party tool? Did we miss options in ADF to handle this?

Answer 1

If I summarize your requirements, you are not just migrating a database to cloud but a complete architecture of your SQL Server, which includes:

1. 1.2 TB of data,
2. Continuous data migration afterwards,
3. Procedures and functions for cross DB queries,
4. Versioned tables

Point 1, 3, and 4 can be done easily by creating and exporting .bacpac file using SQL Server Management Studio (SSMS) from on premises to Azure Blob storage and then importing that file in Azure SQL Database. The .bacpac file that we create in SSMS allows us to include all version tables which we can import at destination database.

Follow this third-party tutorial by sqlshack to migrate data to Azure SQL Database.

The stored procedures can also be moved using SQL Scripts. Follow the below steps:

1. Go the server in Management Studio
2. Select the database, right click on it Go to Task .
3. Select Generate Scripts option under Task
4. Once its started select the desired stored procedures you want to copy and create a file of them and then run script from that file to the Azure SQL DB which you can login in SSMS.

The repeatable migration of data is challenging part. You can try it with Change Data Capture (CDC) but I'm not sure that is what exactly your requirement. You can enable the CDC on database level using below command:

Use <databasename>;
EXEC sys.sp_cdc_enable_db;

Refer to know more - https://www.qlik.com/us/change-data-capture/cdc-change-data-capture#:~:text=Change%20data%20capture%20(CDC)%20refers,a%20downstream%20process%20or%20system .