Azure synapse get metadata

Question

I am trying to get a list of all files in a folder with get metadata activity. To pass this list to the for-each activity, which in turn executes a notebook.

I have a binary dataset and field list is set to child items.

Pipeline crashes every time with the error:

{
    "errorCode": "2011",
    "message": "Blob operation Failed. ContainerName: tmp, path: /tmp/folder/folder1/.",
    "failureType": "UserError",
    "target": "Get Metadata",
    "details": []
}

The files are in 'folder/folder1'.

It's not my first time working with Get Metadata activity and so far it has always worked (in ADF). But I do it first time in Synapse, are there differences? Do you have any ideas what this could be or how I can solve the problem?

Answer 1

Usage of Get Metadata activity to retrieve the metadata of any data is the same in the Azure data factory and Azure Synapse pipeline .

1. Create a binary dataset with a dataset parameter for filename.

2. Connect the binary dataset to the Get metadata activity.

• Pass '*' to the filename parameter value.
• Select child items under Field list to get the list of files/subfolders from the folder.

3. The output gives the list of files from the folder.