Is it possible to have a dynamic strategy matrix in a workflow in GitHub Actions?

Question

I would like to specify a strategy matrix dynamically within the workflow. So, instead of:

strategy:
  matrix:
    foo: [bar, baz]

I want to first call some script which will compute and return an array such as [bar, baz] to me, and then I want to use that as the strategy matrix.

Is this possible?

Answer 1

It's not possible with the available GitHub Actions workflow features but it is possible with a bit hacky solution to provide all the required matrix parameter values' combinations. You can generate all the combinations as a JSON snippet in one of the previous workflow jobs and expose it as the job outputs then use it with matrix include keyword in the next job to provide all the matrix parameters and its values' combinations using fromJson() function as demonstrated in the official announcement . To better explain the concept lets look at the example static matrix job:

jobs:
  matrix-job:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        includes:
          - foo: foo-1
            bar: bar-1
          - foo: foo-1
            bar: bar-2
          - foo: foo-2
            bar: bar-1
    steps:
      - run: |
          echo foo=${{ matrix.foo }}
          echo bar=${{ matrix.bar }}

The workflow outcome is:

In this workflow, all the matrix parameter values combinations are statically provided. We can convert it to be dynamically provided like this:

jobs:
  setup-matrix:
    runs-on: ubuntu-latest
    steps:
      - name: Setup matrix combinations
        id: setup-matrix-combinations
        run: |
          MATRIX_PARAMS_COMBINATIONS='
              {"foo": "foo-1", "bar": "bar-1"},
              {"foo": "foo-1", "bar": "bar-2"},
              {"foo": "foo-2", "bar": "bar-1"},
          '
          echo ::set-output name=matrix-combinations::{\"include\":[$MATRIX_PARAMS_COMBINATIONS]}
    outputs:
      matrix-combinations: ${{ steps.setup-matrix-combinations.outputs.matrix-combinations }}
  matrix-job:
    runs-on: ubuntu-latest
    needs: setup-matrix
    strategy:
      matrix: ${{ fromJson(needs.setup-matrix.outputs.matrix-combinations) }}
    steps:
      - run: |
          echo foo=${{ matrix.foo }}
          echo bar=${{ matrix.bar }}

and the outcome:

The two workflows have equivalent outcomes for the matirx-job but the last one provides a matrix input which is generated dynamically. This is the only way you can dynamically generate a matrix build, you have to provide all the combinations on your own using matrix.include . It's not possible (at the time of writing this) to dynamically provide an array of available values for the given matrix parameter (like in your question) but you have at least dynamic matrix job.

Answer 2

Yes, you can use a matrix that is ingested from an output of a prior step that your step depends on ("needs"):

jobs:
  set:
    runs-on: ubuntu-20.04
    outputs:
      matrix: ${{steps.list_dirs.outputs.matrix}}
    steps:
      - uses: actions/checkout@v2
      - id: list_dirs
        run: echo "::set-output name=matrix::$(ls|jq -cnR '[inputs | select(length>0)]')"

  get:
    runs-on: ubuntu-20.04
    needs: set
    strategy:
      matrix:
        subdir: ${{fromJson(needs.set.outputs.matrix)}}

Answer 3

I have a similar structure as shown above, but I am generating that structure by executing Python code which reads from a Yaml input file. But, somehow, I can't get it working. Since I am using Python script, I need to write the structure (below) to an environment variable before I can assign it to set-output. Then I noticed (after a lot of trials) that I need to put every set in a new line, but it doesn't seem to work. Any help will be appreciated.

 MATRIX_PARAMS_COMBINATIONS='
          {"foo": "foo-1", "bar": "bar-1"},
          {"foo": "foo-1", "bar": "bar-2"},
          {"foo": "foo-2", "bar": "bar-1"},
          '