[英]Azure Data Factory - How to map SQL query results to a JSON string?
I have an SQL table called FileGroups.我有一个名为 FileGroups 的 SQL 表。 I will query for certain columns such as Id, Name and Version.
我将查询某些列,例如 Id、Name 和 Version。 In my Azure Data Factory pipeline, I want to map the column names and row values from the SQL query results to key-value pairs for a JSON string.
在我的 Azure 数据工厂管道中,我想将 SQL 查询结果中的列名和行值映射到 JSON 字符串的键值对。 I also need to include a couple pipeline parameters in the JSON string.
我还需要在 JSON 字符串中包含几个管道参数。 I will then pass this JSON string as input for a stored procedure at the end of my pipeline.
然后,我将把这个 JSON 字符串作为输入传递给管道末端的存储过程。
The resulting JSON string will look like this:生成的 JSON 字符串将如下所示:
{
"id": "guid1",
"name": "fileGroup1",
"version": 1.0,
"pipeline_param_1": "value1",
"pipeline_param_2": "value2"
},
{
"id": "guid2",
"name": "fileGroup2",
"version": 2.0,
"pipeline_param_1": "value1",
"pipeline_param_2": "value2"
}
How do I query the SQL table and construct this JSON string all within my ADF pipeline?如何在我的 ADF 管道中查询 SQL 表并构造这个 JSON 字符串? What activities or data flow transformations do I need to achieve this?
我需要哪些活动或数据流转换来实现这一目标?
the easiest way to implement it is by using a "copy activity"实现它的最简单方法是使用“复制活动”
Here is a quick demo that i created, i want to transform SQL data into Json, i copied SalesLT.Customer data from sql sample data这是我创建的一个快速演示,我想将 SQL 数据转换为 Json,我从 sql 示例数据中复制了 SalesLT.Customer数据
here you can select the columns that you need and add to the data a new column like i did , added a new column " pipId " and used pipeline params.在这里,您可以选择您需要的列并像我一样向数据添加一个新列,添加一个新列“ pipId ”并使用管道参数。
Copy activity in ADF: ADF 中的复制活动:
Data in blob storage: Blob 存储中的数据:
you can read here about copy activity and pipeline params , links:您可以在此处阅读有关复制活动和管道参数的信息,链接:
https://docs.microsoft.com/en-us/azure/data-factory/control-flow-system-variables https://docs.microsoft.com/en-us/azure/data-factory/control-flow-system-variables
https://docs.microsoft.com/en-us/azure/data-factory/copy-activity-overview https://docs.microsoft.com/en-us/azure/data-factory/copy-activity-overview
If your source database is a Microsoft SQL database, like Azure SQL DB, Sql Server, Managed Instance, Azure Synapse Analytics etc, then it is quite capable manipulating JSON.如果您的源数据库是 Microsoft SQL 数据库,如 Azure SQL DB、Sql Server、托管实例、Azure Synapse Analytics 等,那么它非常有能力处理 JSON。 The
FOR JSON
clause constructs valid JSON and you can use options like WITHOUT_ARRAY_WRAPPER
to produce clean output. FOR JSON
子句构造有效的 JSON,您可以使用WITHOUT_ARRAY_WRAPPER
等选项来生成干净的输出。
A simple example:一个简单的例子:
DROP TABLE IF EXISTS #tmp;
CREATE TABLE #tmp (
id VARCHAR(10) NOT NULL,
[name] VARCHAR(20) NOT NULL,
[version] VARCHAR(5) NOT NULL,
pipeline_param_1 VARCHAR(20) NOT NULL,
pipeline_param_2 VARCHAR(20) NOT NULL
);
INSERT INTO #tmp VALUES
( 'guid1', 'fileGroup1', '1.0', 'value1.1', 'value1.2' ),
( 'guid2', 'fileGroup2', '2.0', 'value2.1', 'value2.2' )
SELECT *
FROM #tmp
FOR JSON PATH, WITHOUT_ARRAY_WRAPPER;
Sample output:样本输出:
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.