Azure数据工厂复制活动

Question

我已经为此工作了几天，无法克服此错误。 我在此管道中有2个活动。 第一个活动成功将数据从ODBC连接复制到Azure数据库。 第二个活动将数据从Azure表传输到另一个Azure表，并保持失败。

错误消息是： 复制活动遇到无效的参数：'UnknownParameterName'，详细消息：具有相同键的项已被添加。

我看不到任何无效的参数或未知的参数名称。 我已经使用他们的添加活动代码模板和我自己多次重写了此代码，但是在运行时进行部署时不会收到任何错误。 以下是JSON管道代码。

仅第二活动收到错误。

谢谢。

源数据集

{
"name": "AnalyticsDB-SHIPUPS_06shp-01src_AZ-915PM",
"properties": {
    "structure": [
        {
            "name": "UPSD_BOL",
            "type": "String"
        },
        {
            "name": "UPSD_ORDN",
            "type": "String"
        }
    ],
    "published": false,
    "type": "AzureSqlTable",
    "linkedServiceName": "Source-SQLAzure",
    "typeProperties": {},
    "availability": {
        "frequency": "Day",
        "interval": 1,
        "offset": "04:15:00"
    },
    "external": true,
    "policy": {}
}

}

目标数据集

{
"name": "AnalyticsDB-SHIPUPS_06shp-02dst_AZ-915PM",
"properties": {
    "structure": [
        {
            "name": "SHIP_SYS_TRACK_NUM",
            "type": "String"
        },
        {
            "name": "SHIP_TRACK_NUM",
            "type": "String"
        }
    ],
    "published": false,
    "type": "AzureSqlTable",
    "linkedServiceName": "Destination-Azure-AnalyticsDB",
    "typeProperties": {
        "tableName": "[olcm].[SHIP_Tracking]"
    },
    "availability": {
        "frequency": "Day",
        "interval": 1,
        "offset": "04:15:00"
    },
    "external": false,
    "policy": {}
}

}

管道

{
"name": "SHIPUPS_FC_COPY-915PM",
"properties": {
    "description": "copy shipments ",
    "activities": [
        {
            "type": "Copy",
            "typeProperties": {
                "source": {
                    "type": "RelationalSource",
                    "query": "$$Text.Format('SELECT COMPANY, UPSD_ORDN, UPSD_BOL FROM \"orupsd - UPS interface Dtl\" WHERE COMPANY = \\'01\\'', WindowStart, WindowEnd)"
                },
                "sink": {
                    "type": "SqlSink",
                    "sqlWriterCleanupScript": "$$Text.Format('delete imp_fc.SHIP_UPS_IntDtl_Tracking', WindowStart, WindowEnd)",
                    "writeBatchSize": 0,
                    "writeBatchTimeout": "00:00:00"
                },
                "translator": {
                    "type": "TabularTranslator",
                    "columnMappings": "COMPANY:COMPANY, UPSD_ORDN:UPSD_ORDN, UPSD_BOL:UPSD_BOL"
                }
            },
            "inputs": [
                {
                    "name": "AnalyticsDB-SHIPUPS_03shp-01src_FC-915PM"
                }
            ],
            "outputs": [
                {
                    "name": "AnalyticsDB-SHIPUPS_03shp-02dst_AZ-915PM"
                }
            ],
            "policy": {
                "timeout": "1.00:00:00",
                "concurrency": 1,
                "executionPriorityOrder": "NewestFirst",
                "style": "StartOfInterval",
                "retry": 3,
                "longRetry": 0,
                "longRetryInterval": "00:00:00"
            },
            "scheduler": {
                "frequency": "Day",
                "interval": 1,
                "offset": "04:15:00"
            },
            "name": "915PM-SHIPUPS-fc-copy->[imp_fc]_[SHIP_UPS_IntDtl_Tracking]"
        },
        {
            "type": "Copy",
            "typeProperties": {
                "source": {
                    "type": "SqlSource",
                    "sqlReaderQuery": "$$Text.Format('select distinct ups.UPSD_BOL, ups.UPSD_BOL from imp_fc.SHIP_UPS_IntDtl_Tracking ups LEFT JOIN olcm.SHIP_Tracking st ON ups.UPSD_BOL = st.SHIP_SYS_TRACK_NUM WHERE st.SHIP_SYS_TRACK_NUM IS NULL', WindowStart, WindowEnd)"
                },
                "sink": {
                    "type": "SqlSink",
                    "writeBatchSize": 0,
                    "writeBatchTimeout": "00:00:00"
                },
                "translator": {
                    "type": "TabularTranslator",
                    "columnMappings": "UPSD_BOL:SHIP_SYS_TRACK_NUM, UPSD_BOL:SHIP_TRACK_NUM"
                }
            },
            "inputs": [
                {
                    "name": "AnalyticsDB-SHIPUPS_06shp-01src_AZ-915PM"
                }
            ],
            "outputs": [
                {
                    "name": "AnalyticsDB-SHIPUPS_06shp-02dst_AZ-915PM"
                }
            ],
            "policy": {
                "timeout": "1.00:00:00",
                "concurrency": 1,
                "executionPriorityOrder": "NewestFirst",
                "style": "StartOfInterval",
                "retry": 3,
                "longRetryInterval": "00:00:00"
            },
            "scheduler": {
                "frequency": "Day",
                "interval": 1,
                "offset": "04:15:00"
            },
            "name": "915PM-SHIPUPS-AZ-update->[olcm]_[SHIP_Tracking]"
        }
    ],
    "start": "2017-08-22T03:00:00Z",
    "end": "2099-12-31T08:00:00Z",
    "isPaused": false,
    "hubName": "adf-tm-prod-01_hub",
    "pipelineMode": "Scheduled"
}

}

Answer 1

您看到此链接了吗？

他们得到相同的错误消息，并建议使用AzureTableSink而不是SqlSink

"sink": {
                    "type": "AzureTableSink",
                    "writeBatchSize": 0,
                    "writeBatchTimeout": "00:00:00"
                }

这对您也很有意义，因为您的第二个副本活动是从Azure到Azure

Answer 2

可能是个红色鲱鱼，但我很确定“ tableName”是sqlSource的typeProperties中的必需项。 您的输入数据集缺少此功能。 意识到您在sqlReaderQuery中有一个联接，因此最好在其中放置一个虚拟（但真实）表名。 顺便说一句，如果您不将这些值转换为查询，则不清楚为什么在查询中使用$$ Text.Format和WindowStart / WindowEnd。 您可以将查询放在双引号之间。