Add column to CSV File from another CSV File (Azure Data Factory)

Question

For example:

Persons.csv

name, last_name
-----------------------
jack, jack_lastName
luc, luc_lastname

FileExample.csv

id
243
123

Result:

name, last_name, exampleId
-------------------------------
jack, jack_lastName, 243
luc, luc_lastname, 123

I want to aggregate any number of columns from another data source, to insert that final result in a file or in a database table.

I have been trying many ways but I can't do it.

Answer 1

You can try to make use of Mergefiles in azure data factory pipeline to merge two csv files.

Select copydata activity and go to source to loop through wild card entry *.csv to search for csv files in storage(configure linked storage to adf in this process).

Then the create a output csv in the same container if required as in my case to merge files and store by naming it some examplemerge.csv.

Check mark the first row as header.

validate and try to debug.

Then you must be able to see merged files in the resultant merged file in output folder.

You can check this reference vlog Merge Multiple CSV files to single CSV for more details and also this vlog on Load Multiple CSV Files to a Table in Azure Data Factory if required.

But if you want to join the files, there must be some common column to join. Also check this thread from Q&A Azure Data Factory merge 2 csv files with different schema