[英]Automated Deployment of ADF Pipelines using Azure DevOps CI/CD Pipelines
I have automated the Azure ADF Pipeline Deployment process using Azure DevOps CI/CD pipelines with the help of https://docs.microsoft.com/en-us/azure/data-factory/continuous-integration-deployment (ie) Deploying pipelines from DEV to PROD environment ADF.在https://docs.microsoft.com/en-us/azure/data-factory/continuous-integration-deployment (即)部署管道的帮助下,我使用 Azure DevOps CI/CD 管道自动化了 Azure ADF 管道部署过程从 DEV 到 PROD 环境 ADF。 I am using ARM Templates of the ADF to deploy pipelines from one environment to another.
我正在使用 ADF 的 ARM 模板将管道从一个环境部署到另一个环境。 Hence I will be having a separate ARM_Parameter.json corresponding to each environment(Dev/Prod).
因此,我将有一个单独的 ARM_Parameter.json 对应于每个环境(开发/生产)。 The Problem is each ADF pipeline may have few base parameres along with it, which is not parameterized and hence it will not be available in parameter.json.
问题是每个 ADF 管道可能有很少的基本参数,这些基本参数没有被参数化,因此它在 parameter.json 中不可用。 Can you guys help me to replace the Dev Values with the PROD Values in Base Parameter section under each ADF Pipelines in an automated way during this automated ADF pipeline deployment process using CI/CD Pipelines?
在这个使用 CI/CD 管道的自动化 ADF 管道部署过程中,你们能帮我以自动化的方式在每个 ADF 管道下的基本参数部分用 PROD 值替换开发值吗?
I see two options:我看到两个选项:
You could use Custom parameter with ARM template .您可以将自定义参数与 ARM 模板一起使用。 The custom parameter for Pipeline could look like this:
Pipeline 的自定义参数可能如下所示:
"Microsoft.DataFactory/factories/pipelines": {
"properties": {
"parameters": {
"RUN_ENVIRONMENT": "=:-:string"
}
}
},
Replace the Dev Values with the PROD Values in Base Parameter section
将开发值替换为基本参数部分中的 PROD 值
Based on your screenshot, the RUN_ENVIRONMENT
is the parameter of pipeline, which means while convert to ARM template, its format like:根据您的截图,
RUN_ENVIRONMENT
是管道的参数,这意味着在转换为 ARM 模板时,其格式如下:
"resources": [
{
....
....
"properties": {
"parameters": {
"RUN_ENVIRONMENT": {
"type": "string",
"defaultValue": "pro"
}
},...
},...
}
]
It can not be replaced by using Override template parameters
in ARM deploy task.它不能被 ARM 部署任务中使用
Override template parameters
替换。 Because it will prompt The template parameters 'environment' in the parameters file are not valid; they are not present in the original template and can therefore not be provided at deployment time.
因为会提示
The template parameters 'environment' in the parameters file are not valid; they are not present in the original template and can therefore not be provided at deployment time.
The template parameters 'environment' in the parameters file are not valid; they are not present in the original template and can therefore not be provided at deployment time.
To around this error, just install one extension and add the Replace token
task into the pipeline which before ARM deploy task.要解决此错误,只需安装一个扩展并将
Replace token
任务添加到 ARM 部署任务之前的管道中。 And this task will replace the value of content during the build runtime:此任务将在构建运行时替换 content 的值:
For how to apply this task in our pipeline, you could have a refer to my answer1 and answer2有关如何在我们的管道中应用此任务,您可以参考我的answer1和answer2
There is another approach to publish ADF, from master (collaboration) branch.还有另一种从 master(协作)分支发布 ADF 的方法。 You can define (replace) value for every single node (property) in JSON file (ADF object).
您可以为 JSON 文件(ADF 对象)中的每个节点(属性)定义(替换)值。 It will resolve your problem as you can provide a separate CSV config file per each environment (stage).
它将解决您的问题,因为您可以为每个环境(阶段)提供一个单独的 CSV 配置文件。
Example of CSV config file ( config-stage-UAT.csv
): CSV 配置文件示例 (
config-stage-UAT.csv
):
type,name,path,value
pipeline,PL_CopyMovies,activities[0].outputs[0].parameters.BlobContainer,UAT
Then just run such cmdlet in PowerShell:然后只需在 PowerShell 中运行这样的 cmdlet:
Publish-AdfV2FromJson -RootFolder "$RootFolder" -ResourceGroupName "$ResourceGroupName" -DataFactoryName "$DataFactoryName" -Location "$Location" -Stage "stage-UAT"
Check this out: azure.datafactory.tools (PowerShell module)看看这个: azure.datafactory.tools (PowerShell 模块)
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.