Is it possible to create build pipeline in Azure DevOps for Azure Data factory

Question

I need to automate Azure Data Factory deployment via Azure DevOps. As part of that, I have created a build pipeline but for enabling continuous trigger, which branch should I select? Is it the adf_publish branch? If this is the branch, then I believe a new build should be published manually in the Dev environment to trigger a change in the adf_publish branch, which is a manual process. Thanks in advance!!

Answer 1

You should choose adf_publish branch when enabling continuous trigger. So that whenever changes are published to adf_publish branch, your pipeline will be triggered.

By default, data factory generates the Resource Manager templates of the published factory and saves them into a branch called adf_publish.

To trigger a change in the adf_publish branch, you need to click Publish to manually publish your code changes in the collaboration branch to the Data Factory service, after you have merged changes to the collaboration branch (master is the default). So the the changes will be updated to adf_publish branch.

You can refer to the examples in below blogs:

Continuous integration and delivery in Azure Data Factory

Continuous integration and delivery (CI/CD) in Azure Data Factory using DevOps and GitHub

Azure DevOps Pipeline Setup for Azure Data Factory (v2)

Answer 2

The above mentioned issue where dev values are hardcoded can be changed by adding parametrization template in the Azure DataFactory. Using the below link from Microsoft custom parameters can be added. https://docs.microsoft.com/en-us/azure/data-factory/continuous-integration-deployment#use-custom-parameters-with-the-resource-manager-template

Answer 3

Yes, you can. I am working on a three part series on setting up Azure DevOps for your Data Factory. Check out part 1 which I just published. https://doylestowncoder.com/2021/11/29/building-ci-cd-pipelines-with-azure-data-factory-part-1/