Multi-agent jobs over versions in yaml pipeline

Question

So, I have a yaml pipeline that has an array storing a set of versions in bash, let's say arrayVersions=(3.0.1 3.0.2 ....) .

Now, I want to set up the pipeline that splits each of these versions in one single job in the yaml pipeline, then run them in multi-agent paradigm.

CONTEXT - I have set up the pipeline that iterates over the array and runs, however, it is very slow since it runs sequentially. So, I tried multithreaded parallel programming in bash, but it did not work out. In ideal solution, I am thinking to split up all the versions and run them as a new job in the pipeline. It would be something like this:

jobs:
    # get all the versions
    # split up each version into 1 single job and run the jobs in parallel
    job: 3.0.1
    ...
    job: 3.0.2
    ...

Is there any way I can set it up?

Answer 1

Have you tried using a template and calling it from the jobs section? Here's an example:

# azure-pipelines.yml
trigger:
- none

jobs:
- job: Build
  steps:
  - template: build-specific-version.yml
    parameters:
      appVersion: 
      - '3.0.1'
      - '3.0.2'
      - '3.0.3'

# build-specific-version.yml
parameters:
- name: 'appVersion'
  type: object
  default: 
  - '1.0'
  - '1.1'

steps:
- ${{ each v in parameters.appVersion }}:
  - script: echo ${{ v }}

Docs: Microsoft technical documentation|Template types & usage

Also see: Loops and arrays in Azure Devops Pipelines