[英]What is the ADF v2 equivalent of WindowStart?
我們正在從ADF V1遷移到ADF V2。 我們的管道計划每天運行,並且V2的那部分工作。 管道將在指定的時間開始。
我的問題是如何將計划的觸發器安排為在復制活動中的select語句的where子句中開始的時間? 我懷疑我忽略或誤讀了一些文檔,但是我無法弄清楚。
無論我嘗試什么,該變量似乎都不會在運行時求值。 因此,where子句在運行時看起來像這樣:where last_update> ='@@ pipeline()。TriggerTime'顯然@pipeline()。TriggerTime不是日期。
我已經從https://docs.microsoft.com/zh-cn/azure/data-factory/control-flow-system-variables嘗試了以下參數列表
這是在V1中起作用的:“ sqlReaderQuery”:“ $$ Text.Format('從表中選擇列,其中last_update> = \\'{0:yyyy-MM-dd} \\'',WindowStart,WindowEnd)”
我嘗試過的事情:
-將@trigger()。scheduledTime作為觸發器中的參數傳遞給where子句中的@ {formatDateTime(pipeline()。parameters.startDate,'yyyy-MM-dd')},在管道中將參數定義為字符串類型的startDate。
-將@pipeline()。TriggerTime設置為pipeline()。parameters.startDate的默認值。
-在where子句中調用@pipeline()。TriggerTime。
提前謝謝了。
編輯:
{
"name": "PL_ADLS_RAW_IDMTables_DAILY",
"properties": {
"activities": [
{
"name": "isStartDateNotNull",
"description": "If the startDate parameter == 0 then run the full load. If the startDate parameter != 0 then run for >= startDate parameter.",
"type": "IfCondition",
"typeProperties": {
"expression": {
"value": "@equals(pipeline().parameters.startDate,'0')",
"type": "Expression"
},
"ifFalseActivities": [
{
"name": "copy_ENTITY_FULL",
"description": "Copies ENTITY from IDM Database to ADLS Raw Zone",
"type": "Copy",
"policy": {
"timeout": "7.00:00:00",
"retry": 0,
"retryIntervalInSeconds": 60,
"secureOutput": false,
"secureInput": false
},
"typeProperties": {
"source": {
"type": "SqlSource",
"sqlReaderQuery": {
"value": "SELECT [ENTITY_ID],[ENTITY_TYPE_ID],REPLACE(REPLACE([SHORT_NAME], CHAR(13),' '), CHAR(10),' '),REPLACE(REPLACE([ENTITY_NAME], CHAR(13),' '), CHAR(10),' '),REPLACE(REPLACE([ENTITY_DESCRIPTION], CHAR(13),' '), CHAR(10),' '),REPLACE(REPLACE([STATUS], CHAR(13),' '), CHAR(10),' '),[STATUS_DATE],[REVISION_ID],[CREATED_BY],[CREATION_DATE],[LAST_UPDATED_BY],[LAST_UPDATE_DATE],REPLACE(REPLACE([COMMENTS], CHAR(13),' '), CHAR(10),' ') FROM [dbo].[ENTITY] where LAST_UPDATE_DATE >= '@{formatDateTime(pipeline().parameters.startDate,'yyyy-MM-dd')}'",
"type": "Expression"
}
},
"sink": {
"type": "AzureDataLakeStoreSink"
},
"enableStaging": false,
"dataIntegrationUnits": 0
},
"inputs": [
{
"referenceName": "DS_IN_SQL_IDM_ENTITY",
"type": "DatasetReference"
}
],
"outputs": [
{
"referenceName": "DS_OUT_ADLS_RAW_IDM_ENTITY",
"type": "DatasetReference"
}
]
}
],
"ifTrueActivities": [
{
"name": "copy_ENTITY",
"description": "Copies ENTITY from IDM Database to ADLS Raw Zone",
"type": "Copy",
"policy": {
"timeout": "7.00:00:00",
"retry": 0,
"retryIntervalInSeconds": 60,
"secureOutput": false,
"secureInput": false
},
"typeProperties": {
"source": {
"type": "SqlSource",
"sqlReaderQuery": {
"value": "SELECT [ENTITY_ID],[ENTITY_TYPE_ID],REPLACE(REPLACE([SHORT_NAME], CHAR(13),' '), CHAR(10),' '),REPLACE(REPLACE([ENTITY_NAME], CHAR(13),' '), CHAR(10),' '),REPLACE(REPLACE([ENTITY_DESCRIPTION], CHAR(13),' '), CHAR(10),' '),REPLACE(REPLACE([STATUS], CHAR(13),' '), CHAR(10),' '),[STATUS_DATE],[REVISION_ID],[CREATED_BY],[CREATION_DATE],[LAST_UPDATED_BY],[LAST_UPDATE_DATE],REPLACE(REPLACE([COMMENTS], CHAR(13),' '), CHAR(10),' ') FROM [dbo].[ENTITY]",
"type": "Expression"
}
},
"sink": {
"type": "AzureDataLakeStoreSink"
},
"enableStaging": false,
"dataIntegrationUnits": 0
},
"inputs": [
{
"referenceName": "DS_IN_SQL_IDM_ENTITY",
"type": "DatasetReference"
}
],
"outputs": [
{
"referenceName": "DS_OUT_ADLS_RAW_IDM_ENTITY",
"type": "DatasetReference"
}
]
}
]
}
}
],
"parameters": {
"startDate": {
"type": "String"
}
}
},
"type": "Microsoft.DataFactory/factories/pipelines"
}
{
"name": "TR_SCHED_0800EST",
"properties": {
"description": "Daily 0800 EST",
"runtimeState": "Stopped",
"pipelines": [
{
"pipelineReference": {
"referenceName": "PL_ADLS_RAW_IDMTables_DAILY",
"type": "PipelineReference"
},
"parameters": {
"startDate": "@trigger().scheduledTime"
}
}
],
"type": "ScheduleTrigger",
"typeProperties": {
"recurrence": {
"frequency": "Day",
"interval": 1,
"startTime": "2018-08-30T13:00:00Z",
"timeZone": "UTC",
"schedule": {
"minutes": [
0
],
"hours": [
13
]
}
}
}
}
}
編輯
這有效
這行不通
聲明:本站的技術帖子網頁,遵循CC BY-SA 4.0協議,如果您需要轉載,請注明本站網址或者原文地址。任何問題請咨詢:yoyou2525@163.com.