ADF（azure数据工厂）将数据从SQL Server复制到Cosmos DB，其中字段包含json对象

Question

我有SQL数据库包含表之间的一对多关系。 我编写了一个查询，以便它包含包含json（对于相关表行）的字段的行。 这是查询 -

select msg.MessageId as id
      ,msg.CreatedDate as [CreatedDate]
      ,
      (select [RecipientTypeId] as [RecipientTypeId]
      ,[RecipientId] as [RecipientId]
      ,[mr.CreatedDate] as [CreatedDate]
      ,[IsRead] as [IsRead]
      ,[ReadDate] as [ReadDate]
       from [dbo].[MsgRecipients] mr
       where msg.messageid = mr.messageid  FOR JSON PATH, INCLUDE_NULL_VALUES) as Recipients
       ,
       (select 
       [Data] as [Data]
      ,[Value] as [Value]
      ,[mc.SomeId] as [SomeId]
       from [dbo].[MessageContent] mc
       where msg.messageid = mc.messageid  FOR JSON PATH, WITHOUT_ARRAY_WRAPPER, INCLUDE_NULL_VALUES) as MessageContent
       from [dbo].[Messages] msg

以下是查询结果 -

这里我得到两个字段，Recipients包含对象数组，Messages包含对象集。

在ADF我写了这个 -

{
    "name": "CopyPipeline_SQL_to_Cosmos",
    "properties": {
        "description": "CopyPipeline_SQL_to_Cosmos",
        "activities": [
            {
                "name": "Copy_lbp",
                "type": "Copy",
                "policy": {
                    "timeout": "7.00:00:00",
                    "retry": 0,
                    "retryIntervalInSeconds": 30,
                    "secureOutput": false,
                    "secureInput": false
                },
                "userProperties": [
                    {
                        "name": "Destination",
                        "value": "Messages1"
                    }
                ],
                "typeProperties": {
                    "source": {
                        "type": "AzureSqlSource",
                        "sqlReaderQuery": {
                            "value": "select msg.MessageId as id, msg.CreatedDate as [CreatedDate], (select [RecipientTypeId] as [RecipientTypeId], [RecipientId] as [RecipientId], [mr.CreatedDate] as [CreatedDate], [ReadDate] as [ReadDate] from [dbo].[MsgRecipients] mr where msg.messageid = mr.messageid  FOR JSON PATH, INCLUDE_NULL_VALUES) as Recipients, (select  [Data] as [Data], [Value] as [Value], [mc.SomeId] as [SomeId] from [dbo].[MessageContent] mc where msg.messageid = mc.messageid  FOR JSON PATH, WITHOUT_ARRAY_WRAPPER, INCLUDE_NULL_VALUES) as MessageContent from [dbo].[Messages] msg  where CreatedDate >= '@{formatDateTime(pipeline().parameters.windowStart, 'yyyy-MM-dd HH:mm' )}' AND CreatedDate < '@{formatDateTime(pipeline().parameters.windowEnd, 'yyyy-MM-dd HH:mm' )}'",
                            "type": "Expression"
                        }
                    },
                    "sink": {
                        "type": "DocumentDbCollectionSink",
                        "nestingSeparator": "",
                        "writeBatchSize": 10000,
                        "writeBehavior": "upsert"
                    },
                    "enableStaging": false
                },
                "inputs": [
                    {
                        "referenceName": "SourceDataset_lbp",
                        "type": "DatasetReference"
                    }
                ],
                "outputs": [
                    {
                        "referenceName": "DestinationDataset_lbp",
                        "type": "DatasetReference"
                    }
                ]
            },
            {
                "name": "Custom1",
                "type": "Custom",
                "policy": {
                    "timeout": "7.00:00:00",
                    "retry": 0,
                    "retryIntervalInSeconds": 30,
                    "secureOutput": false,
                    "secureInput": false
                }
            }
        ],
        "parameters": {
            "windowStart": {
                "type": "String"
            },
            "windowEnd": {
                "type": "String"
            }
        }
    },
    "type": "Microsoft.DataFactory/factories/pipelines"
}

我正在获取记录 - >

在上面的图像中，您可以注意到在Recipients和MessageContent中它添加了整个json。

虽然我期待这个 -

Answer 1

我没有方便的数据，所以我无法测试，但我认为你将不得不使用“架构映射”。

请阅读这里的结构https://docs.microsoft.com/en-us/azure/data-factory/copy-activity-schema-and-type-mapping#alternative-schema-mapping