[英]Store values from an array of JSON objects in SQL DB using Azure Data Factory
I am trying to teach myself Azure Data Factory and am attempting some "simple" API exercises.我正在尝试自学 Azure 数据工厂并尝试一些“简单”的 API 练习。
I successfully loaded data from an API call into a SQL database using the copy data
activity and adding the simple mapping.我使用
copy data
活动并添加简单映射成功地将来自 API 调用的数据加载到 SQL 数据库中。
However, this was for an API call that returned a single JSON object as below.但是,这是针对返回单个 JSON object 的 API 调用,如下所示。
{
"id": "tt0110413",
"title": "Léon: The Professional",
"year": "1994",
"releaseDate": "1994-11-18",
"runtimeMins": "110",
"plot": "12-year-old Mathilda is reluctantly taken in by Léon...",
"imDbRating":"8.5"
}
I would like to progress this to be able to handle an array of JSON objects as below.我想对此进行改进,以便能够处理 JSON 对象数组,如下所示。
{
"items":
[
{
"id": "tt0111161",
"title": "The Shawshank Redemption"
},
{
"id": "tt0068646",
"title": "The Godfather"
},
{
"id": "tt0468569",
"title": "The Dark Knight"
}
],
"errorMessage": ""
}
I have naively thought this would be a simple matter pointing the copy data
activity at the new JSON and updating the mapping as required.我天真地认为这将是一件简单的事情,
copy data
活动指向新的 JSON 并根据需要更新映射。
However I now receive the following error:但是我现在收到以下错误:
Data type of column 'id' can't be inferred from 1st row of data, please specify its data type in mappings of copy activity or structure of DataSet
无法从第一行数据中推断出“id”列的数据类型,请在复制活动或数据集结构的映射中指定其数据类型
Would someone be able to advise on how I change the mapping to handle this, or if in fact this is entirely the wrong approach.有人可以建议我如何更改映射来处理这个问题,或者事实上这完全是错误的方法。
As always, any help is greatly appreciated.与往常一样,非常感谢任何帮助。
Since you only have a one level array, this shouldn't be hard.由于您只有一个一级数组,因此这应该不难。 You will need to use a collection reference, and in your example that would be "items".
您将需要使用集合引用,在您的示例中,这将是“项目”。
Unfortunately, ADF is very limited on the complexity of collections/arrays in JSON and can only handle a single collection reference.不幸的是,ADF 在 JSON 中的集合/数组的复杂性方面非常有限,并且只能处理单个集合引用。 I have found it personally easier to do a WEB API call instead of a copy activity and push that JSON text to a Stored Procedure.
我发现个人更容易执行 WEB API 调用而不是复制活动并将 JSON 文本推送到存储过程。
From there you can pretty easily shred the JSON however you would like.从那里你可以很容易地切碎 JSON,但是你想。 Here is an example of what a pretty simple Stored Proc would look like that takes the JSON Output:
这是一个非常简单的存储过程的示例,它采用 JSON Output:
ALTER Procedure [Log].[PopulatePowerBIGroups] (
@json NVARCHAR(MAX))
as
select @json j
;
insert into Log.PowerBI_GroupsTmp
SELECT
JSON_VALUE ( j.[value], '$.id' ) AS groupID,
JSON_VALUE ( j.[value], '$.isReadOnly' ) AS isReadOnly,
JSON_VALUE ( j.[value], '$.isOnDedicatedCapacity' ) AS isOnDedicatedCapacity,
JSON_VALUE ( j.[value], '$.capacityId' ) AS capacityId,
JSON_VALUE ( j.[value], '$.capacityMigrationStatus' ) AS capacityMigrationStatus,
JSON_VALUE ( j.[value], '$.description' ) AS description,
JSON_VALUE ( j.[value], '$.type' ) AS type,
JSON_VALUE ( j.[value], '$.state' ) AS state,
JSON_VALUE ( j.[value], '$.name' ) AS groupName,
Stag.CSTReturnDate(getdate()) as ETL_UpdateTimestamp
FROM OPENJSON( @json ) j
where JSON_VALUE ( j.[value], '$.name' ) not like 'DUMMY%'
and JSON_VALUE ( j.[value], '$.state' ) = 'Active'
But of course for very simple web output, a Copy Data activity will work just fine!当然,对于非常简单的 web output,复制数据活动就可以正常工作!
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.