在 Docker 中为 Android 运行并行 Gitlab CI 作业

Question

I use Gitlab CI to build my Android app using fastlane inside a Docker container. My app has two "flavors", which I want to build in separate CI jobs. Here is the relevant portion of the .gitlab-ci.yml :

default:
  image: registry.example.com/group/project:29-android-fastlane-debian
  tags:
    - docker
  before_script:
    - ruby -v # Print out ruby version for debugging
    - bundle install

build_flavor1_debug:
  stage: build
  script:
    - bundle exec fastlane build release:"false" --env flavor1
  artifacts:
    paths:
      - app/build/outputs/bundle/
  rules:
      - if: '$CI_COMMIT_TAG && $CI_COMMIT_REF_PROTECTED == "true"'
        when: never
      - when: on_success

build_flavor2_debug:
  stage: build
  script:
    - bundle exec fastlane build release:"false" --env flavor2
  artifacts:
    paths:
      - app/build/outputs/bundle/
  rules:
      - if: '$CI_COMMIT_TAG && $CI_COMMIT_REF_PROTECTED == "true"'
        when: never
      - when: on_success

(Other jobs in this pipeline do release builds for git tags and upload to the Play Store, but those are not relevant to this issue.)

Here are the relevant parts of the Fastfile :

# All flavors we know about. If one is not specified, run them all
flavors = {
  "flavor1" => "com.example.flavor1.app",
  "flavor2" => "com.example.flavor2.app"
}

# If set to a single flavor, make it the only one in the array
if ENV["FLAVOR_NAME"]
  flavors = flavors.slice(ENV["FLAVOR_NAME"])
end

UI.user_error!("Unknown flavor '#{ENV["FLAVOR_NAME"]}' selected") if flavors.empty?

platform :android do

  desc "Build the application"
  lane :build do |options|
    setup(options)

    flavors.each { |flavor_name, flavor_package|
      build_flavor(
        flavor: flavor_name,
        release: options[:release]
      )
    }
  end

end

Using the --env flag loads the .env file with the appropriate FLAVOR_NAME variable set to make only one flavor run at a time.

This worked fine when I was running the builds sequentially, but that takes way too long. I changed the Gitlab Runner configuration to allow running up to 8 simultaneous jobs, and now I get the following error:

The message received from the daemon indicates that the daemon has disappeared.
Build request sent: Build{id=111291aa-90bc-45c3-8fb4-9a271d4663f4, currentDir=/builds/group/project}
Attempting to read last messages from the daemon log...
Daemon pid: 1100
  log file: /root/.gradle/daemon/6.5/daemon-1100.out.log
----- Last  20 lines from daemon log file - daemon-1100.out.log -----

[SNIP] - No useful logs here...

----- End of the daemon log -----
FAILURE: Build failed with an exception.
* What went wrong:
Gradle build daemon disappeared unexpectedly (it may have been killed or may have crashed)

Sometimes this happens on one flavor's job and sometimes on the other. If I manually rerun the failed job, it always succeeds.

I believe that what is happening is that the two build jobs are running inside the same Docker container instead of each one having its own. Whichever one finishes using Gradle first shuts down the Gradle daemon which causes the other one to fail.

How can I get these jobs to be able to run in parallel?

Answer 1

Do you have enough resources for simultaneous runs? Especially memory can be a problem, so OS will be killing another containers due to memory pressure.