您如何使用 azure dev Ops 并行运行 xcuitests？

Question

I'd like to execute my xcuitest suite on azure dev ops in parallel using xcode's out of the box capabilities. Locally, it's as simple as checking the box to enable parallel testing on my test target.

Parallel executions toggle

Locally, xcode opens multiple simulators and the tests run as expected. On azure, they run and pass as expected but they take the same amount of time as they usually do, indicating to me that they are not running in parallel. What am I missing here? Are there extra steps I need to take to get them running in parallel via azure dev ops?

Azure-Pipleline.yml snippet

 - stage: uiTests
displayName: UI Tests
dependsOn: []
condition: | # don't run for release builds to save Azure bandwith
  not(
    or(
      contains(variables['Build.SourceBranch'], 'master'), 
      contains(variables['Build.SourceBranch'], 'release')
    )
  )
jobs:
  - template: azure-templates/job-tests.yml
    parameters:
      configuration: "$(UI_TEST_CONFIGURATION)"
      sdk: "$(UI_TEST_SDK)"
      scheme: "$(UI_TEST_SCHEME)"
      artifactName: "UI Tests Results - Attempt #$(System.StageAttempt).$(System.JobAttempt)" # include attempt number in suffix because reruns can not overwrite artifacts"
      shouldRunWiremock: true

azure-templates/job-test.yml snippet

 - job: Test
pool: Hosted macOS
variables:
  - template: variables.yml
  - group: blah-Keys
steps:
  - template: steps-install-code-signing.yml
    parameters:
      certFile: "$(UAT_SIGNING_FILE_ID)"
      signingIdentity: "$(UAT_SIGNING_IDENTITY)"
      provisioningFile: "$(UAT_PROVISIONING_FILE_ID)"
  - script: "./setBuildVersion.sh"
    displayName: "Update Build Number"
  - script: "./xcconfig.sh"
    displayName: "Populate xcconfig files"
  - bash: "./runWiremock.sh &"
    continueOnError: true
    displayName: "Start Mock Server"
    enabled: ${{ parameters.shouldRunWiremock }}
  - task: Xcode@5
    displayName: "Run Tests"
    inputs:
      actions: test
      configuration: "${{ parameters.configuration }}"
      sdk: "${{ parameters.sdk }}"
      xcWorkspacePath: "$(WORKSPACE_PATH)"
      scheme: "${{ parameters.scheme }}"
      xcodeVersion: specifyPath
      xcodeDeveloperDir: "$(XCODE_PATH)"
      signingOption: default
      args: "-resultBundlePath $(build.SourcesDirectory)/testResults"
      destinationPlatformOption: iOS
      destinationSimulators: "$(TEST_SIMULATOR)"
      publishJUnitResults: true
    continueOnError: false # report test failures as errors in Azure DevOps
  - publish: testResults
    artifact: "${{ parameters.artifactName }}.xcresult"
    condition: not(canceled()) # run if there are test failures

Answer 1

You can consider to use build matrix which is one feature of YAML to achieve the test ran with multiple simulators:

jobs:
- job: Build
  strategy:
    matrix:
      simulators1:
        destinationSimulators: 'iPhone X'
      simulators2:
        destinationSimulators: 'iPhone 7'


..
..
- task: Xcode@5
   displayName: "Run Tests"
..
..
  destinationSimulators: "$(destinationSimulators)"
..
..