ADF（azure数据工厂）将数据从SQL Server复制到Cosmos DB，其中字段包含json对象

Question

I have SQL Database contain one to many relations between tables. I wrote a query so that it contains rows with fields contains json (for related table rows). Here is the query -

select msg.MessageId as id
      ,msg.CreatedDate as [CreatedDate]
      ,
      (select [RecipientTypeId] as [RecipientTypeId]
      ,[RecipientId] as [RecipientId]
      ,[mr.CreatedDate] as [CreatedDate]
      ,[IsRead] as [IsRead]
      ,[ReadDate] as [ReadDate]
       from [dbo].[MsgRecipients] mr
       where msg.messageid = mr.messageid  FOR JSON PATH, INCLUDE_NULL_VALUES) as Recipients
       ,
       (select 
       [Data] as [Data]
      ,[Value] as [Value]
      ,[mc.SomeId] as [SomeId]
       from [dbo].[MessageContent] mc
       where msg.messageid = mc.messageid  FOR JSON PATH, WITHOUT_ARRAY_WRAPPER, INCLUDE_NULL_VALUES) as MessageContent
       from [dbo].[Messages] msg

Following is the results of query -

Here I am getting two fields with Recipients contains array of objects and Messages contains set of object.

In ADF I wrote this -

{
    "name": "CopyPipeline_SQL_to_Cosmos",
    "properties": {
        "description": "CopyPipeline_SQL_to_Cosmos",
        "activities": [
            {
                "name": "Copy_lbp",
                "type": "Copy",
                "policy": {
                    "timeout": "7.00:00:00",
                    "retry": 0,
                    "retryIntervalInSeconds": 30,
                    "secureOutput": false,
                    "secureInput": false
                },
                "userProperties": [
                    {
                        "name": "Destination",
                        "value": "Messages1"
                    }
                ],
                "typeProperties": {
                    "source": {
                        "type": "AzureSqlSource",
                        "sqlReaderQuery": {
                            "value": "select msg.MessageId as id, msg.CreatedDate as [CreatedDate], (select [RecipientTypeId] as [RecipientTypeId], [RecipientId] as [RecipientId], [mr.CreatedDate] as [CreatedDate], [ReadDate] as [ReadDate] from [dbo].[MsgRecipients] mr where msg.messageid = mr.messageid  FOR JSON PATH, INCLUDE_NULL_VALUES) as Recipients, (select  [Data] as [Data], [Value] as [Value], [mc.SomeId] as [SomeId] from [dbo].[MessageContent] mc where msg.messageid = mc.messageid  FOR JSON PATH, WITHOUT_ARRAY_WRAPPER, INCLUDE_NULL_VALUES) as MessageContent from [dbo].[Messages] msg  where CreatedDate >= '@{formatDateTime(pipeline().parameters.windowStart, 'yyyy-MM-dd HH:mm' )}' AND CreatedDate < '@{formatDateTime(pipeline().parameters.windowEnd, 'yyyy-MM-dd HH:mm' )}'",
                            "type": "Expression"
                        }
                    },
                    "sink": {
                        "type": "DocumentDbCollectionSink",
                        "nestingSeparator": "",
                        "writeBatchSize": 10000,
                        "writeBehavior": "upsert"
                    },
                    "enableStaging": false
                },
                "inputs": [
                    {
                        "referenceName": "SourceDataset_lbp",
                        "type": "DatasetReference"
                    }
                ],
                "outputs": [
                    {
                        "referenceName": "DestinationDataset_lbp",
                        "type": "DatasetReference"
                    }
                ]
            },
            {
                "name": "Custom1",
                "type": "Custom",
                "policy": {
                    "timeout": "7.00:00:00",
                    "retry": 0,
                    "retryIntervalInSeconds": 30,
                    "secureOutput": false,
                    "secureInput": false
                }
            }
        ],
        "parameters": {
            "windowStart": {
                "type": "String"
            },
            "windowEnd": {
                "type": "String"
            }
        }
    },
    "type": "Microsoft.DataFactory/factories/pipelines"
}

I am getting records as this ->

In above image, You can notice that in Recipients and MessageContent it has added whole json.

Whereas I am expecting this -

Answer 1

I don't have the data handy so I cannot test , but I am think that you will have to use the "Schema mapping" .

Please read about structure here https://docs.microsoft.com/en-us/azure/data-factory/copy-activity-schema-and-type-mapping#alternative-schema-mapping