Transform JSON Data of one column from excel csv in Azure data factory

Question

Have an excel csv as a source in that one particular entity (500) contains JSON formatted data with hierarchy like below

500 7   30600052764 30128903357 {"type": "reportingData", "merStoreId": "0099703", "productData": [{"productCode": "107", "totalAmount": "47.92", "quantity": "1.000", "unitPrice": "47.92", "tax1Amount": "0.00"},{"productCode": "963", "totalAmount": "2.40", "quantity": "1.000", "unitPrice": "2.40", "tax1Amount": "0.00"}, {"productCode": "913", "totalAmount": "20.00", "quantity": "1.000", "unitPrice": "20.00", "tax1Amount": "0.00"}]}

Has to convert this data in excel csv to my sink in meaningful format, can any one help me

Answer 1

To unroll the Json column as you want you can follow below steps in Data flow activity:

• Sample data at source

• As it is taking the Json column as string first, I took Parse transformation and unfold the Json. In parse setting select Document form as single document and column as name of column you want to give Expression as column you want to unfold and expression as

(type as string,        merStoreId as integer,      productData as string[])

Data preview:

• After that create derived column transformation and create column with the respective value of column unfolded in previous transformationData preview:

-As you have one object with array in Json column to unfold that take flatten transformation and flattern that array column.

• Now, unfold that array Json in another parse transformation parse setting select Document form as single document and column as name of column you want to give Expression as column you want to unfold and expression as

(productCode as integer,        totalAmount as double,      quantity as double,     unitPrice as double,        tax1Amount as double)

Data preview:

• After that create derived column transformation and create column with the respective value of column unfolded in previous transformationData preview:

• Now take union transformation after derived column 1 and with derived column 2Data preview:

• now take select transformation and select the columns and store the columns into the sink.