I have a source JSON file with hierarchical data, which I need to sink in two SQL tables(relational). The JSON is as below
{
"orders":[
{
"orderid":"30933",
"ordername":"abc",
"items":[
{
"itemid":1,
"itemstatus":"Failed"
},
{
"itemid":2,
"itemstatus":"Failed"
}
]
},
{
"orderid":"308320",
"ordername":"xyz",
"items":[
{
"itemid":5,
"itemstatus":"Succeeded"
}
]
}
]
}
My SQL holding two tables Order and OrderItem with OrderID primary and foreign key.
Now I have an Azure data factory data flow with source as above JSON and I need to park all data relational in respective tables.
So here I need OrderId(30933,308320) and OrderName(abc,xyz) will go into Order table and respective items data go into OrderItem table(which reference OrderId from Order table). In this case Order table have 2 and OrderItem table have 3 entries.
We can not achieve that in one copy active.
We could using two copy actives in one pipeline, I tested and it succeed. You could follow my steps bellow:
Orders
(orderid and ordername) to table Orders
.items
(itemid and itemstatus) to table OrderItems
.Note:
Copy active 1 and 2 use the same json file as the source. The differences are in the Mapping settings.
Copy active 1 sink is Azure SQL database table Orders
, Copy active 2 sink is Azure SQL database table OrderItems
.
To make you understand it clearly, I made two GIF pictures.
Mapping settings in Copy active 1:
Mapping settings in Copy active 2:
Run the pipeline:
Check the data in table:
The limit is that we only could get the first element of items
, we can not choose the collection reference both.
Update:
Congratulations that you achieved it in another way: flatten/transpose the data using data flow and then maintain it to pour in a relational SQL table.
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.