将数据从本地DW迁移到Azure

Question

我有一个使用SQL Server的本地Dat Warehouse，将数据加载到SQL Data Warehouse的最佳方法是什么？

Answer 1

The process of loading data depends on the amount of data. For very small data sets (<100 GB) you can simply use the bulk copy command line utility (bcp.exe) to export the data from SQL Server and then import to Azure SQL Data Warehouse. For data sets greater than 100 GB, you can export your data using bcp.exe, move the data to Azure Blob Storage using a tool like AzCopy , create an external table (via TSQL code) and then pull the data in via a Create Table As Select (CTAS) statement.

Using the PolyBase/CTAS route will allow you to take advantage of multiple compute nodes and the parallel nature of data processing in Azure SQL Data Warehouse - an MPP based system. This will greatly improve the data ingestion performance as each compute node is able to process a block of data in parallel with the other nodes.

One consideration as well is to increase the amount of DWU (compute resources) available in SQL Data Warehouse at the time of the CTAS statement. This will increase the number of compute resources adding additional parallelism which will decrease the total ingestion time.

Answer 2

SQL database migration wizard is a helpful tool to migrate schema and data from an on-premise database to Azure sql databases. http://sqlazuremw.codeplex.com/