Can you download a Qualtrics data file to blob storage using Azure Data Factory?

Question

Qualtrics provides an API that allows you to download your survey data as a file. The API call looks like this:

https://[tenant].qualtrics.com/API/v3/surveys/[Survey_ID]/export-responses/[File_ID]/file

The response contains ALL the data (in .csv format) and must be written straight to a file (ie responses.csv)in blob storage for further processing.

I've tried using a web activity and a copy activity (w/ Rest) with no luck. Any help greatly appreciated.

Here is what a sample result response from the API call looks like:

RecordedDate,RecipientFirstName,RecipientLastName,CSULBID,Score
2020-02-07 23:21:50,Darth,Vader,00001234,56
2020-02-07 23:21:50,Darth,Maul,00005678,56

Answer 1

Have you looked into Azure Logic Apps?

It has a very intuitive way to call web API and create CSV out of it. It's explained here: https://docs.microsoft.com/en-us/azure/logic-apps/logic-apps-perform-data-operations

My group tried to use Azure Data Factory, but it seemed like Azure Logic App did a better job, and ended up creating the API calling and CSV making step as a Logic App.