Powershell script for building CI/CD for Azure Data factory

Question

I'm planning to build Continous integration and Deployment for Azure Data factory using PowerShell. So all the build and release process which can be done using VSTS has to be done using Powershell. If anyone can share any links or powershell scripts it would be helpful. Thanks

Answer 1

If you mean create build/release definition through PowerShell, you can do it through VSTS REST with PowerShell, simple code:

$base64AuthInfo = [Convert]::ToBase64String([Text.Encoding]::ASCII.GetBytes(("{0}:{1}" -f "","{personal access token}")))
$body='{
    "variables": {
        "system.debug": {
            "value": "false",
            "allowOverride": true
        }
    },
    "retentionRules": [
        {
            "branches": [
                "+refs/heads/*"
            ],
            "artifacts": [],
            "artifactTypesToDelete": [
                "FilePath",
                "SymbolStore"
            ],
            "daysToKeep": 10,
            "minimumToKeep": 1,
            "deleteBuildRecord": true,
            "deleteTestResults": true
        }
    ],
    "properties": {},
    "tags": [],
    "jobAuthorizationScope": "projectCollection",
    "jobTimeoutInMinutes": 60,
    "jobCancelTimeoutInMinutes": 5,
    "process": {
        "phases": [
            {
                "steps": [
                    {
                        "environment": {},
                        "enabled": true,
                        "continueOnError": false,
                        "alwaysRun": false,
                        "displayName": "Azure PowerShell script: FilePath",
                        "timeoutInMinutes": 0,
                        "condition": "succeeded()",
                        "task": {
                            "id": "72a1931b-effb-4d2e-8fd8-f8472a07cb62",
                            "versionSpec": "2.*",
                            "definitionType": "task"
                        },
                        "inputs": {
                            "ConnectedServiceNameSelector": "ConnectedServiceNameARM",
                            "ConnectedServiceName": "",
                            "ConnectedServiceNameARM": "aad18bbf-0333-41bf-99ea-674181d17ada",
                            "ScriptType": "FilePath",
                            "ScriptPath": "$/Scrum2015/ClassLibraryA/dbtest.ps1",
                            "Inline": "# You can write your azure powershell scripts inline here. \n# You can also pass predefined and custom variables to this script using arguments",
                            "ScriptArguments": "",
                            "TargetAzurePs": "LatestVersion",
                            "CustomTargetAzurePs": ""
                        }
                    }
                ],
                "name": "Phase 1",
                "refName": "Phase_1",
                "condition": "succeeded()",
                "target": {
                    "executionOptions": {
                        "type": 0
                    },
                    "allowScriptsAuthAccessOption": false,
                    "type": 1
                },
                "jobAuthorizationScope": "projectCollection",
                "jobCancelTimeoutInMinutes": 1
            }
        ],
        "type": 1
    },
    "repository": {
        "properties": {
            "cleanOptions": "0",
            "tfvcMapping": "{\"mappings\":[{\"serverPath\":\"$/Scrum2015/ClassLibraryA\",\"mappingType\":\"map\",\"localPath\":\"\\\\\"},{\"serverPath\":\"$/Scrum2015/ClassLibraryA/VS2010UltimTrial.iso\",\"mappingType\":\"cloak\",\"localPath\":\"\"},{\"serverPath\":\"$/Scrum2015/ClassLibraryA/tools\",\"mappingType\":\"cloak\",\"localPath\":\"\"}]}",
            "labelSources": "0",
            "labelSourcesFormat": "$(build.buildNumber)"
        },
        "id": "$/",
        "type": "TfsVersionControl",
        "name": "Scrum2015",
        "defaultBranch": "$/Scrum2015/ClassLibraryA",
        "rootFolder": "$/Scrum2015",
        "clean": "false",
        "checkoutSubmodules": false
    },
    "processParameters": {},
    "quality": "definition",
    "drafts": [],
    "queue": { 
        "id": 179
    },

    "name": "PowerShellRest2VNext",
    "path": "\\",
    "type": "build",
    "queueStatus": "enabled"
}'
$result = Invoke-RestMethod -Uri $uri -Method Post -ContentType "application/json" -Headers @{Authorization=("Basic {0}" -f $base64AuthInfo)} -Body $body

Note, you can create a sample build/release definition, then get it through VSTS REST API to get JSON content.

A blog that can help you: VSTS/TFS REST API: The basics and working with builds and releases

If you mean custom build/release task, you can refer to Add a build or release task , for powershell, change execution like this:

"execution": {
    "PowerShell3": {
      "target": "Hello.ps1",
      "argumentFormat": ""
    }
}

Regarding deploy azure data factory, you can refer to: How to deploy Azure Data Factory pipeline and its dependencies programatically using PowerShell (Do not need to call Login-AzureRMAccount and Select-AzureRMSubscrition , just add Azure PowerShell task to build/release definition).

A 3rd extension: Azure Data Factory