[英]Lookup activity in an Azure Data Factory Data Copy Source - Syntax Error
通过https://docs.microsoft.com/zh-cn/azure/data-factory/tutorial-incremental-copy-portal进行工作
我使用sqlReaderQuery进行了一个名为GetCurrentWatermarkValue的查找活动:
Select WaterMarkValue as CurrentWatermarkValue\nfrom WatermarkTable
我使用sqlReaderQuery进行了另一个名为GetNewWatermarkValue的活动:
select max(createdon) as NewWatermarkValue from shipment
然后,我尝试使用以下两种方式在源代码中同时使用它们:
select *
from Shipment
where CreatedOn > '@{activity('GetCurrentWatermarkValue').output.firstRow.CurrentWatermarkValue}'
and CreatedOn <= '@{activity('GetNewWatermarkValue').output.firstRow.NewWatermarkValue}'
预览数据按钮为灰色(但是当我删除where条件时启用),因此明显存在错误
设置接收器后,我尝试设置映射。 单击“映射”选项卡上的“导入模式”,将得到:
A database operation failed with the following error: 'Incorrect syntax near 'GetCurrentWatermarkValue'.'. Activity ID:98794aa9-c866-48d6-b9ff-9cb277bac6ed
我想也许我应该使用添加动态内容选项,但这只是给
Query is required
我在某处阅读了在Lookup活动上设置“仅第一行”时,firstRow之后的文本。 应该是[TableName],但这似乎不正确。
抬头:
{
"name": "GetCurrentWatermarkValue",
"type": "Lookup",
"policy": ...,
"typeProperties": {
"source": {
"type": "AzureSqlSource",
"sqlReaderQuery": "Select WaterMarkValue as CurrentWatermarkValue\nfrom WatermarkTable"
},
"dataset": {
"referenceName": "WatermarkTable",
"type": "DatasetReference"
}
}
}
抬头:
{
"name": "GetNewWatermarkValue",
"type": "Lookup",
"policy": ...,
"typeProperties": {
"source": {
"type": "AzureSqlSource",
"sqlReaderQuery": "select max(createdon) as NewWatermarkValue from shipment"
},
"dataset": {
"referenceName": "ShipmentsTable",
"type": "DatasetReference"
}
}
}
资料复制:
{
"name": "ArchiveShipments",
"type": "Copy",
"dependsOn": [
{
"activity": "GetCurrentWatermarkValue",
"dependencyConditions": [
"Succeeded"
]
},
{
"activity": "GetNewWatermarkValue",
"dependencyConditions": [
"Succeeded"
]
}
],
"policy": ...,
"typeProperties": {
"source": {
"type": "AzureSqlSource",
"sqlReaderQuery": {
"value": "select *\nfrom Shipment\nwhere CreatedOn > '@{activity('GetCurrentWatermarkValue').output.firstRow.CurrentWatermarkValue}' \nand CreatedOn <= '@{activity('GetNewWatermarkValue').output.firstRow.NewWatermarkValue}'",
"type": "Expression"
}
},
"sink": {
"type": "AzureSqlSink"
},
"enableStaging": false
},
"inputs": [
{
"referenceName": "ShipmentsTable",
"type": "DatasetReference"
}
],
"outputs": [
{
"referenceName": "ShipmentArchiveTable",
"type": "DatasetReference"
}
]
}
这看起来像是语法问题,请尝试以下为复制活动查询:
@{CONCAT('select * from Shipment where CreatedOn > ', activity('GetCurrentWatermarkValue').output.firstRow.CurrentWatermarkValue, ' and CreatedOn <= ', activity('GetNewWatermarkValue').output.firstRow.NewWatermarkValue}
除非要手动设置映射,否则不要使用映射,如果列名相同,则只需单击“清除”。
希望这对您有所帮助!
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.