Docker 已激活 Conda Env 并准备好执行 Shell 命令的图像

Question

I have tried many ways through searching for a solution.

I think my problem is different.

I am wanting to have a docker image that has the environment installed and then active and ready for shell commands like: flake8, pylint, black, isort, coverage

Dockerfile

FROM continuumio/miniconda3

# Create the environment:
COPY conda_env_unit_tests.yml .
RUN conda env create -f conda_env_unit_tests.yml
RUN echo "conda activate up-and-down-pytorch" >> ~/.bashrc

conda_env_unit_test.yml

name: up-and-down-pytorch
channels:
  - defaults
  - conda-forge
  
dependencies:
  - python=3.9
  - pytest
  - pytest-cov
  - black
  - flake8
  - isort
  - pylint

.gitlab-ci.yml (slimmed down)

stages:
  - docker
  - linting
  - test

build_unit_test_docker:
  stage: docker
  tags:
    - docker
  image: docker:stable
  services:
    - docker:dind
  variables:
    IMAGE_NAME: "miniconda3-up-and-down-unit-tests"
  script:
    - cp /builds/upanddown1/mldl/up_and_down_pytorch/conda_env_unit_tests.yml /builds/upanddown1/mldl/up_and_down_pytorch/docker/unit_tests/
    - docker -D login $CI_REGISTRY -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD
    - docker -D build -t $CI_REGISTRY/upanddown1/mldl/up_and_down_pytorch/$IMAGE_NAME docker/unit_tests/
    - docker -D push $CI_REGISTRY/upanddown1/mldl/up_and_down_pytorch/$IMAGE_NAME
  rules:
    - changes:
      - docker/unit_tests/Dockerfile
      - conda_env_unit_tests.yml

unit-test:
  stage: test
  # image: continuumio/miniconda3:latest
  image: $CI_REGISTRY/upanddown1/mldl/up_and_down_pytorch/miniconda3-up-and-down-unit-tests
  script:
    # - conda env create --file conda_env.yml
    # - source activate up-and-down-pytorch
    - coverage run --source=. -m pytest --verbose
    - coverage report
    - coverage xml
  coverage: '/(?i)total.*? (100(?:\.0+)?\%|[1-9]?\d(?:\.\d+)?\%)$/'
  artifacts:
    reports:
      coverage_report:
        coverage_format: cobertura
        path: coverage.xml

The Docker Image gets uploaded to the gitlab registry and the unit test stage uses that image, however:

/bin/bash: line 127: coverage: command not found

(ultimate goal was to not have to create the conda environment every time I wanted to lint or run unit tests)

Answer 1

Figured it out today. Dropped the duration from 8.5 minutes to 1:07 for the unit tests.

Change was to source the environment in the unit-test job. Didn't need to do that in the Dockerfile.

Dockerfile

FROM continuumio/miniconda3

# Create the environment:
COPY conda_env_unit_tests.yml .
RUN conda env create -f conda_env_unit_tests.yml

conda_env_unit_tests.yml

name: up-and-down-pytorch
channels:
  - defaults
  - conda-forge
  
dependencies:
  - python=3.9
  - pandas
  - pytest
  - pytest-cov
  - black
  - flake8
  - isort
  - pylint

.gitlab-ci.yml (slimmed down)

stages:
  - docker
  - linting
  - test

build_unit_test_docker:
  stage: docker
  tags:
    - docker
  image: docker:stable
  services:
    - docker:dind
  variables:
    IMAGE_NAME: "miniconda3-up-and-down-unit-tests"
  script:
    - cp /builds/upanddown1/mldl/up_and_down_pytorch/conda_env_unit_tests.yml /builds/upanddown1/mldl/up_and_down_pytorch/docker/unit_tests/
    - docker -D login $CI_REGISTRY -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD
    - docker -D build -t $CI_REGISTRY/upanddown1/mldl/up_and_down_pytorch/$IMAGE_NAME docker/unit_tests/
    - docker -D push $CI_REGISTRY/upanddown1/mldl/up_and_down_pytorch/$IMAGE_NAME
  rules:
    - changes:
      - docker/unit_tests/Dockerfile
      - conda_env_unit_tests.yml

unit-test:
  stage: test
  image: $CI_REGISTRY/upanddown1/mldl/up_and_down_pytorch/miniconda3-up-and-down-unit-tests
  script:
    - source activate up-and-down-pytorch
    - coverage run --source=. -m pytest --verbose
    - coverage report
    - coverage xml
  coverage: '/(?i)total.*? (100(?:\.0+)?\%|[1-9]?\d(?:\.\d+)?\%)$/'
  artifacts:
    reports:
      coverage_report:
        coverage_format: cobertura
        path: coverage.xml