[英]Create a table in Azure SQL Database from Blob Storage
I need to take large tables out of our Azure Data Warehouse and move them over to stand alone Azure SQL Databases. 我需要从Azure数据仓库中取出大表,然后将它们移到独立的Azure SQL数据库中。 I haven't been able to get the Data Factory to work quickly enough for my scenario.
我无法使数据工厂在我的场景中足够快地工作。 I can get my tables into Blob storage from my Data Warehouse via external tables.
我可以通过外部表从我的数据仓库中获取我的表到Blob存储。 What I can not figure out is how to create an external table on an Azure SQL Database with an external data source to my Blob storage.
我无法弄清楚的是如何使用外部数据源在Azure SQL数据库上创建外部表到我的Blob存储。
This is the format file, external data source, and external table used to get my table into blob storage: 这是用于将表格放入blob存储的格式文件,外部数据源和外部表:
CREATE EXTERNAL FILE FORMAT [DelimitedText]
WITH (
FORMAT_TYPE = DELIMITEDTEXT,
FORMAT_OPTIONS (
FIELD_TERMINATOR = N'~¶~',
USE_TYPE_DEFAULT = False
),
DATA_COMPRESSION = N'org.apache.hadoop.io.compress.GzipCodec')
GO
CREATE EXTERNAL DATA SOURCE [myDataSource]
WITH (
TYPE = HADOOP,
LOCATION = N'wasbs://<blob container>@<storage account>.blob.core.windows.net',
CREDENTIAL = [myCredential])
GO
CREATE EXTERNAL TABLE [dbo].[myTable]
WITH (
DATA_SOURCE = [myDataSource] ,
LOCATION = N'MY_FOLDER/',
FILE_FORMAT = [DelimitedText]
)
AS
SELECT *
FROM dbo.mytable
The only external data source I'm able to create in the Azure SQL Database is of TYPE=SHARD_MAP_MANAGER
is that right or necessary? 我能够在Azure SQL数据库中创建的唯一外部数据源是
TYPE=SHARD_MAP_MANAGER
是正确还是必要? This link looks like I should be able to create an external data source using TYPE=HADOOP
but I get an "error near EXTERNAL" error. 这个链接看起来我应该能够使用
TYPE=HADOOP
创建一个外部数据源,但是我得到一个“EXTERNAL附近的错误”错误。 I'm also unable to create an EXTERNAL FILE FORMAT. 我也无法创建外部文件格式。 Is that possible in Azure SQL Database?
这可能在Azure SQL数据库中吗?
https://msdn.microsoft.com/en-us/library/dn935022.aspx#Examples: Azure SQL Database https://msdn.microsoft.com/en-us/library/dn935022.aspx#Examples:Azure SQL数据库
Ultimately, I'm trying to create an external table to my blob storage and then insert into a table in my Azure SQL Database from that blob. 最后,我正在尝试为我的blob存储创建一个外部表,然后从该blob插入到我的Azure SQL数据库中的表中。 Then drop the container.
然后放下容器。
It is not possible to use PolyBase features on Azure SQL Database, only in on-premise SQL Server 2016 databases. 无法在Azure SQL数据库上使用PolyBase功能,仅在本地 SQL Server 2016数据库中使用。
In the article there is a note: 在文章中有一个注释:
PolyBase is supported only on SQL Server 2016, Azure SQL Data Warehouse, and Parallel Data Warehouse.
仅在SQL Server 2016,Azure SQL数据仓库和并行数据仓库上支持PolyBase。 Elastic Database queries are supported only on Azure SQL Database v12 or later.
仅在Azure SQL数据库v12或更高版本上支持弹性数据库查询。
Instead you could create an Azure SQL Data Warehouse (on the same Azure SQL Server if you wish). 相反,您可以创建Azure SQL数据仓库 (如果您愿意,可以在同一个Azure SQL Server上)。 The guide will work for you if you run on that instead.
如果您继续使用该指南,该指南将适合您。 It will not work for Hadoop ( https://msdn.microsoft.com/en-us/library/mt703314.aspx ), but as I understand your question you are importing from an Azure blob storage, and that will work on Azure SQL data warehouse.
它不适用于Hadoop( https://msdn.microsoft.com/en-us/library/mt703314.aspx ),但据我了解您的问题,您是从Azure blob存储导入的,这将适用于Azure SQL数据仓库。
Azure SQL Database has recently gained the ability to load files from Azure Blob Storage using either BULK INSERT or OPENROWSET. Azure SQL数据库最近获得了使用BULK INSERT或OPENROWSET从Azure Blob存储加载文件的功能。 Start here .
从这里开始。
Two simple code examples taken from the linked article: 从链接文章中获取的两个简单代码示例:
BULK INSERT Product
FROM 'data/product.dat'
WITH ( DATA_SOURCE = 'MyAzureBlobStorageAccount');
SELECT Color, count(*)
FROM OPENROWSET(BULK 'data/product.bcp', DATA_SOURCE = 'MyAzureBlobStorage',
FORMATFILE='data/product.fmt', FORMATFILE_DATA_SOURCE = 'MyAzureBlobStorage') as data
GROUP BY Color;
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.