What is the correct/idiomatic way of handling complex JSON data in Logic Apps?
Question
I am reading data from a web service using a JSON api call. It returns data in (IMHO) an unusual format:
{
"statusCode": 200,
"headers": {
"Transfer-Encoding": "chunked",
"Content-Type": "application/json; charset=UTF-8",
},
"body": {
"columns": {
"column": [
{
"id": -1,
"val": null,
"name": "COLUMNA",
"caption": "Column A",
"value": null
},
{
"id": 106009714,
"val": null,
"name": "COLUMNB",
"caption": "Column B",
"value": null
},
{
"id": 106010949,
"val": null,
"name": "COLUMNC",
"caption": "Column C",
"value": null
},
{
"id": 106009719,
"val": null,
"name": "COLUMND",
"caption": "Column D",
"value": null
},
"records": {
"record": [
{
"field": [
{
"id": -1,
"val": "390011768",
"name": "COLUMNA",
"caption": null,
"value": null
},
{
"id": 106009714,
"val": "Lorem ipsum",
"name": "COLUMNB",
"caption": null,
"value": null
},
{
"id": 106010949,
"val": "Nulla mauris ante",
"name": "COLUMNC",
"caption": null,
"value": null
},
{
"id": 106009719,
"val": "15-01-2019 12:46",
"name": "COLUMND",
"caption": null,
"value": null
}
]
},
{
"field": [
{
"id": -1,
"val": "390012438",
"name": "COLUMNA",
"caption": null,
"value": null
},
{
"id": 106009714,
"val": "Vestibulum ligula",
"name": "COLUMNB",
"caption": null,
"value": null
},
{
"id": 106010949,
"val": "Nulla elit orci",
"name": "COLUMNC",
"caption": null,
"value": null
},
{
"id": 106009719,
"val": "27-03-2019 14:17",
"name": "COLUMND",
"caption": null,
"value": null
}
]
},
{
"field": [
{
"id": -1,
"val": "390013343",
"name": "COLUMNA",
"caption": null,
"value": null
},
{
"id": 106009714,
"val": "Nunc magna risus",
"name": "COLUMNB",
"caption": null,
"value": null
},
{
"id": 106010949,
"val": "Vivamus rutrum",
"name": "COLUMNC",
"caption": null,
"value": null
},
{
"id": 106009719,
"val": "23-07-2019 13:17",
"name": "COLUMND",
"caption": null,
"value": null
}
]
}
]
}
}
}
The first block describes the column headings and is followed by another block where each row is defined as an array of fields. My goal is to convert this to the following (IMHO) more usual format:
{
{
"Column A":"390011768",
"Column B":"Lorem ipsum",
"Column C":"Nulla mauris ante",
"Column D":"27-03-2019 14:17"
},
etc
}
I have come up with a solution but it feels messy/wrong and involves lots of conversions between strings objects and arrays. The code for this is below:
{
"definition": {
"$schema": "https://schema.management.azure.com/providers/Microsoft.Logic/schemas/2016-06-01/workflowdefinition.json#",
"actions": {
"CreateJSON": {
"inputs": {
"body": {
"columns": {
"column": [
{
"caption": "Column A",
"id": -1,
"name": "COLUMNA",
"val": null,
"value": null
},
{
"caption": "Column B",
"id": 106009714,
"name": "COLUMNB",
"val": null,
"value": null
},
{
"caption": "Column C",
"id": 106010949,
"name": "COLUMNC",
"val": null,
"value": null
},
{
"caption": "Column D",
"id": 106009719,
"name": "COLUMND",
"val": null,
"value": null
}
]
},
"records": {
"record": [
{
"field": [
{
"caption": null,
"id": -1,
"name": "COLUMNA",
"val": "390011768",
"value": null
},
{
"caption": null,
"id": 106009714,
"name": "COLUMNB",
"val": "Lorem ipsum",
"value": null
},
{
"caption": null,
"id": 106010949,
"name": "COLUMNC",
"val": "Nulla mauris ante",
"value": null
},
{
"caption": null,
"id": 106009719,
"name": "COLUMND",
"val": "15-01-2019 12:46",
"value": null
}
]
},
{
"field": [
{
"caption": null,
"id": -1,
"name": "COLUMNA",
"val": "390012438",
"value": null
},
{
"caption": null,
"id": 106009714,
"name": "COLUMNB",
"val": "Vestibulum ligula",
"value": null
},
{
"caption": null,
"id": 106010949,
"name": "COLUMNC",
"val": "Nulla elit orci",
"value": null
},
{
"caption": null,
"id": 106009719,
"name": "COLUMND",
"val": "27-03-2019 14:17",
"value": null
}
]
},
{
"field": [
{
"caption": null,
"id": -1,
"name": "COLUMNA",
"val": "390013343",
"value": null
},
{
"caption": null,
"id": 106009714,
"name": "COLUMNB",
"val": "Nunc magna risus",
"value": null
},
{
"caption": null,
"id": 106010949,
"name": "COLUMNC",
"val": "Vivamus rutrum",
"value": null
},
{
"caption": null,
"id": 106009719,
"name": "COLUMND",
"val": "23-07-2019 13:17",
"value": null
}
]
}
]
}
},
"headers": {
"Content-Type": "application/json; charset=UTF-8",
"Date": "Tue, 22 Sep 2020 17:26:18 GMT",
"Transfer-Encoding": "chunked"
},
"statusCode": 200
},
"runAfter": {},
"type": "Compose"
},
"Create_HTML_table": {
"inputs": {
"format": "HTML",
"from": "@variables('RecordArray')"
},
"runAfter": {
"ReconstituteArray": [
"Succeeded"
]
},
"type": "Table"
},
"FlattenArray": {
"inputs": {
"variables": [
{
"name": "ArrayString",
"type": "string",
"value": "@{replace(join(variables('RowAccumulator'), ','),'],[',',')}"
}
]
},
"runAfter": {
"RowLoop": [
"Succeeded"
]
},
"type": "InitializeVariable"
},
"HeaderCollection": {
"actions": {
"ColumnHeadings": {
"inputs": {
"caption": "@items('HeaderCollection')['caption']",
"name": "@items('HeaderCollection')['name']"
},
"runAfter": {},
"type": "Compose"
}
},
"foreach": "@outputs('CreateJSON')['body']['columns']['column']",
"runAfter": {
"CreateJSON": [
"Succeeded"
]
},
"type": "Foreach"
},
"InitFieldAccumulator": {
"inputs": {
"variables": [
{
"name": "FieldAccumulator",
"type": "array"
}
]
},
"runAfter": {
"InitRowString": [
"Succeeded"
]
},
"type": "InitializeVariable"
},
"InitRowAccumulator": {
"inputs": {
"variables": [
{
"name": "RowAccumulator",
"type": "array"
}
]
},
"runAfter": {
"InitFieldAccumulator": [
"Succeeded"
]
},
"type": "InitializeVariable"
},
"InitRowString": {
"inputs": {
"variables": [
{
"name": "RowString",
"type": "string"
}
]
},
"runAfter": {
"HeaderCollection": [
"Succeeded"
]
},
"type": "InitializeVariable"
},
"ReconstituteArray": {
"inputs": {
"variables": [
{
"name": "RecordArray",
"type": "array",
"value": "@json(variables('ArrayString'))"
}
]
},
"runAfter": {
"FlattenArray": [
"Succeeded"
]
},
"type": "InitializeVariable"
},
"RowLoop": {
"actions": {
"FieldLoop": {
"actions": {
"Compose": {
"inputs": "@setProperty(item(), 'caption', body('MatchCaption')[0]['caption'])",
"runAfter": {
"MatchCaption": [
"Succeeded"
]
},
"type": "Compose"
},
"FieldAccumulator": {
"inputs": {
"name": "FieldAccumulator",
"value": "@outputs('Compose')"
},
"runAfter": {
"Compose": [
"Succeeded"
]
},
"type": "AppendToArrayVariable"
},
"MatchCaption": {
"inputs": {
"from": "@outputs('ColumnHeadings')",
"where": "@equals(item()['name'], items('FieldLoop')['name'])"
},
"runAfter": {},
"type": "Query"
}
},
"foreach": "@items('RowLoop')['field']",
"runAfter": {},
"runtimeConfiguration": {
"concurrency": {
"repetitions": 1
}
},
"type": "Foreach"
},
"FlattenRow": {
"inputs": {
"name": "RowString",
"value": "@{replace(string(body('Select')), '},{', ',')}"
},
"runAfter": {
"Select": [
"Succeeded"
]
},
"type": "SetVariable"
},
"Reset_FieldAccumulator": {
"inputs": {
"name": "FieldAccumulator",
"value": []
},
"runAfter": {
"RowAccumulator": [
"Succeeded"
]
},
"type": "SetVariable"
},
"RowAccumulator": {
"inputs": {
"name": "RowAccumulator",
"value": "@variables('RowString')"
},
"runAfter": {
"FlattenRow": [
"Succeeded"
]
},
"type": "AppendToArrayVariable"
},
"Select": {
"inputs": {
"from": "@variables('FieldAccumulator')",
"select": {
"@{item()['caption']}": "@item()['val']"
}
},
"runAfter": {
"FieldLoop": [
"Succeeded"
]
},
"type": "Select"
}
},
"foreach": "@outputs('CreateJSON')['body']['records']['record']",
"runAfter": {
"InitRowAccumulator": [
"Succeeded"
]
},
"runtimeConfiguration": {
"concurrency": {
"repetitions": 1
}
},
"type": "Foreach"
}
},
"contentVersion": "1.0.0.0",
"outputs": {},
"parameters": {},
"triggers": {
"Recurrence": {
"recurrence": {
"frequency": "Day",
"interval": 1
},
"type": "Recurrence"
}
}
},
"parameters": {}
}
First I create a JSON payload (mocking what comes back from the API). Then I have a loop which creates an array with the column names and captions, which I use later to provide the friendly column names.
The second loop iterates over the 'record' elements which themselves are arrays. It uses FilterArray to find the correct caption, uses Compose to create a JSON fragment and then stores this for each field.
When all the fields for a row have been processed, I use Select to isolate the values I need and then transform this into a 'normal' JSON element and store that.
After all rows are processed I perform some more processing, worked out by trial and error, to derive something that will work with the Create HTML Table function.
While this approach works, it seems too complex. However I couldn't find another way to do this that worked. I am new to Logic Apps but not programming. Can anyone provide a better/more idiomatic way of achieving this? Or is this type of processing best outsourced to a function?
PS The Logic App will operate on small datasets (double digits) so I am not too bothered by performance but presumably a more idiomatic solution would perform better.
1 answers
solution1
1 2020-10-06 12:58:01
You can use Liquid templates to define complex json transformations, as suggested by the docs :
For advanced and complex JSON to JSON transformations that have elements such as iterations, control flows, and variables, create and use templates that describe these transformations by using the Liquid open-source template language.
An easy way to create liquid templates is using Visual Studio Code with some plugins .
The main benefit is that authoring the transformation this way is much easier dan using the Logic App Designer. Also, the transformation logic is seperated from the business logic.
You can also use Liquid templates to transform the JSON to text, so transforming to html directly is an option.
By the way, it seems your input json is invalid, so I couldn't provide a simple example.
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.