将Blob数据复制到本地sql表中

Question

My problem statement is that I have a csv blob and I need to import that blob into a sql table. Is there an utility to do that?

I was thinking of one approach, that first to copy blob to on-premise sql server using AzCopy utility and then import that file in sql table using bcp utility. Is this the right approach? and I am looking for 1-step solution to copy blob to sql table.

Answer 1

Regarding your question about the availability of a utility which will import data from blob storage to a SQL Server, AFAIK there's none. You would need to write one.

Your approach seems OK to me. Though you may want to write a batch file or something like that to automate the whole process. In this batch file, you would first download the file on your computer and the run the BCP utility to import the CSV in SQL Server. Other alternatives to writing batch file are:

1. Do this thing completely in PowerShell.
2. Write some C# code which makes use of storage client library to download the blob and once the blob is downloaded, start the BCP process in your code.

Answer 2

To pull a blob file into an Azure SQL Server, you can use this example syntax (this actually works, I use it):

BULK INSERT MyTable 
FROM 'container/folder/folder/file' 
WITH ( DATA_SOURCE = 'ds_blob',BATCHSIZE=10000,FIRSTROW=2);

• MyTable has to have identical columns (or it can be a view against a table that yields identical columns)
• In this example, ds_blob is an external data source which needs to be created beforehand ( https://docs.microsoft.com/en-us/sql/t-sql/statements/create-external-data-source-transact-sql )
• The external data source needs to use a database contained credential, which uses an SAS key which you need to generate beforehand from blob storage https://docs.microsoft.com/en-us/sql/t-sql/statements/create-database-scoped-credential-transact-sql )

The only downside to this mehod is that you have to know the filename beforehand - there's no way to enumerate them from inside SQL Server.

I get around this by running powershell inside Azure Automation that enumerates blobds and writes them into a queue table beforehand