[英]Azure Data factory copy activity failed mapping strings (from csv) to Azure SQL table sink uniqueidentifier field
I have an Azure data factory (DF) pipeline that consists a Copy activity.我有一个包含复制活动的 Azure 数据工厂 (DF) 管道。 The Copy activity uses HTTP connector as source to invoke a REST end-point and returns csv stream that sinks with Azure SQL Database table.
复制活动使用 HTTP 连接器作为源来调用 REST 端点并返回与 Azure SQL 数据库表一起接收的 csv 流。
The Copy fails when CSV contains strings (such as 40f52caf-e616-4321-8ea3-12ea3cbc54e9
) which are mapped to an uniqueIdentifier field in target table with error message The given value of type String from the data source cannot be converted to type uniqueidentifier of the specified target column
.当 CSV 包含映射到目标表中的 uniqueIdentifier 字段的字符串(例如
40f52caf-e616-4321-8ea3-12ea3cbc54e9
)时,复制失败并显示错误消息The given value of type String from the data source cannot be converted to type uniqueidentifier of the specified target column
。
I have tried to wrapped the source string with {}
such as {40f52caf-e616-4321-8ea3-12ea3cbc54e9}
with no success.我试图用
{}
包装源字符串,例如{40f52caf-e616-4321-8ea3-12ea3cbc54e9}
没有成功。
The Copy activity will work if I modified the target table field from uniqueIdentifier
to nvarchar(100)
.如果我将目标表字段从
uniqueIdentifier
修改为nvarchar(100)
,则 Copy 活动将起作用。
I reproduce your issue on my side.我在我这边重现您的问题。
The reason is data types of source and sink are dismatch.You could check the Data type mapping for SQL server .原因是 source 和 sink 的数据类型不匹配。你可以检查SQL server 的数据类型映射。
Your source data type is string
which is mapped to nvarchar or varchar, and uniqueidentifier
in sql database needs GUID
type in azure data factory.您的源数据类型是映射到 nvarchar 或 varchar 的
string
,sql 数据库中的uniqueidentifier
需要 azure 数据工厂中的GUID
类型。
So,please configure sql server stored procedure
in your sql server sink as a workaround.因此,请在您的 sql server sink 中配置 sql server
stored procedure
作为解决方法。
Please follow the steps from this doc :请按照此文档中的步骤操作:
Step 1: Configure your Sink dataset:第 1 步:配置您的接收器数据集:
Step 2: Configure Sink section in copy activity as follows:第 2 步:在复制活动中配置 Sink 部分,如下所示:
Step 3: In your database, define the table type with the same name as sqlWriterTableType.第 3 步:在您的数据库中,定义与 sqlWriterTableType 同名的表类型。 Notice that the schema of the table type should be same as the schema returned by your input data.
请注意,表类型的架构应与输入数据返回的架构相同。
CREATE TYPE [dbo].[CsvType] AS TABLE(
[ID] [varchar](256) NOT NULL
)
Step 4: In your database, define the stored procedure with the same name as SqlWriterStoredProcedureName.第 4 步:在您的数据库中,定义与 SqlWriterStoredProcedureName 同名的存储过程。 It handles input data from your specified source, and merge into the output table.
它处理来自您指定源的输入数据,并合并到输出表中。 Notice that the parameter name of the stored procedure should be the same as the "tableName" defined in dataset.
请注意,存储过程的参数名称应与数据集中定义的“tableName”相同。
Create PROCEDURE convertCsv @ctest [dbo].[CsvType] READONLY
AS
BEGIN
MERGE [dbo].[adf] AS target
USING @ctest AS source
ON (1=1)
WHEN NOT MATCHED THEN
INSERT (id)
VALUES (convert(uniqueidentifier,source.ID));
END
Output:输出:
Hope it helps you.Any concern,please free feel to let me know.希望它可以帮助你。任何问题,请随时让我知道。
There is a way to fix guid conversion into uniqueidentifier
SQL column type properly via JSON configuration.有一种方法可以通过 JSON 配置正确地将 guid 转换为
uniqueidentifier
SQL 列类型。 Edit the Copy Activity via Code {} button in top right toolbar.编辑右上角工具栏中的通过代码复制活动 {} 按钮。
Put:放:
"translator": {
"type": "TabularTranslator",
"typeConversion": true
}
into typeProperties
block of the Copy
activity.进入
Copy
活动的typeProperties
块。 This will also work if Mapping schema is unspecified / dynamic.如果映射模式未指定/动态,这也将起作用。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.