[英]SSIS: How can you enumerate through files in Azure Storage and load each file into SQL Server
I would like to do something that seems straightforward, but I am not having much joy.我想做一些看似简单的事情,但我并没有太多的快乐。 I want to build an SSIS Package using the For Each Loop container configured for the Foreach Azure Blob enumerator to loop through the files in my storage container.
我想使用为 Foreach Azure Blob 枚举器配置的 For Each Loop 容器构建一个 SSIS Package 以遍历我的存储容器中的文件。 I can get the loop to indeed loop through all the files.
我可以让循环确实循环遍历所有文件。 The problem comes when I try to load each file using the 'Azure Blob Source' data flow component.
当我尝试使用“Azure Blob Source”数据流组件加载每个文件时,问题就出现了。 It seems as if the 'Azure Blob Source' data flow component is not able to be provided with the blob name that should come from the Foreach Azure Blob enumerator.
似乎无法为“Azure Blob 源”数据流组件提供应该来自 Foreach Azure Blob 枚举器的 blob 名称。 This results in the 'Azure Blob Source' data flow component loading the same file that is set as its 'Blob Name' over and over as the enumerator loops through all the files in the container.
这会导致“Azure Blob 源”数据流组件在枚举器循环遍历容器中的所有文件时一遍又一遍地加载设置为其“Blob 名称”的同一文件。 Surely it should be possible to provide a variable as the 'Blob Name' so that the package can call the Data Flow task for each file in the container.
当然,应该可以提供一个变量作为“Blob 名称”,以便 package 可以为容器中的每个文件调用数据流任务。
I solved this by doing the following.我通过执行以下操作解决了这个问题。 Firstly you have to get the Blob Name from the Azure Blob enumerator stored in a package scoped variable.
首先,您必须从 Azure Blob 枚举器中获取 Blob 名称,该枚举器存储在 package 范围变量中。
I created a user variable called FilePath which is loaded with each blob name in the blob container as it loops through them.我创建了一个名为 FilePath 的用户变量,它在 blob 容器中循环加载每个 blob 名称。 Within the foreach loop container I put a data flow task that has an 'Azure Blob Source' component in it.
在 foreach 循环容器中,我放置了一个包含“Azure Blob Source”组件的数据流任务。 The Azure Blob Source component will load data from a specified text file.
Azure Blob Source 组件将从指定的文本文件加载数据。 When you set up the component you specify the name an example text file in the format you are loading in order to define how it is imported.
当您设置组件时,您以您正在加载的格式指定示例文本文件的名称,以便定义它的导入方式。 When the 'Azure Blob Source' component is located within the Azure Blob enumerator you would want it to be provided with the name of each blob in the container as it enumerates through them.
当“Azure Blob 源”组件位于 Azure Blob 枚举器中时,您可能希望在枚举容器中为它提供每个 blob 的名称。
You do that by setting up an Expression at the Data Flow component level.您可以通过在数据流组件级别设置表达式来做到这一点。 You need to set up a conditional formula to provide a value for the 'Blob Name' even when the enumerator is not in operation or you will have and error saying there is no value for the Blob Name.
您需要设置一个条件公式来为“Blob 名称”提供一个值,即使枚举器未在运行,否则您将遇到错误,指出 Blob 名称没有值。
@[User::FilePath] == "" ? "example.dat" : @[User::FilePath]
The logic states that if the user variable FilePath has no value assigned, just use a fake filename to avoid the error about having no Blob Name in the Azure Blob Source, however if the enumerator is providing a name then use that as provided in User::FilePath逻辑表明,如果用户变量 FilePath 没有分配值,只需使用假文件名以避免 Azure Blob 源中没有 Blob 名称的错误,但是如果枚举器提供名称,则使用用户中提供的名称: :文件路径
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.