减小 docker 构建大小，在两个图像之间共享 conda 环境

Question

I'm trying to build webapp using AWS. I've got a docker-compose.yml that builds two images; a service image (running a flask server script) and a worker image (doing all the calculations sent to it from the flask server).

services:
  worker:
    image: co2gasp/worker:latest
    build: ./worker_app
  web:
    image : co2gasp/service:latest 
    build: ./server_app

The problem I'm having is that I'm running into errors particularly with memory when building the containers and I want to keep the build as small as possible. In short the Docker compose file is identical for both images (see below) and it builds two identical conda environments in the two separate images – when it builds the second image it's running out of memory. What I'm wondering is if there is anyway to build a single environment and share it between both images?

FROM continuumio/miniconda3

RUN apt-get update -y
RUN apt-get install zip -y
RUN apt-get install awscli -y
WORKDIR /app
## Create the environment:
COPY environment.yml .
#Make RUN commands use the new environment:
RUN conda env create -f environment.yml

COPY ./PHREEQC /PHREEQC
COPY ./service /service
COPY ./temp_files /temp_files
COPY ./INPUT_DATA /INPUT_DATA
COPY ./PHREEQC/phreeqc_files/database/pitzer.dat /bin/pitzer.dat
COPY ./PHREEQC/phreeqc_files/bin/phreeqc /bin/phreeqc
ENV PATH=${PATH}:/bin/phreeqc
ENV PATH=${PATH}:/bin/pitzer.dat
ENV PATH=${PATH}:/bin
RUN echo 'Adding new'
RUN echo "conda activate myenv" >> ~/.bashrc

SHELL ["conda", "run", "-n", "myenv", "/bin/bash", "-c"]

# Demonstrate the environment is activated:
RUN echo "Make sure flask is installed:"
RUN python -c "import flask"

RUN echo "Copy service directory"

WORKDIR /service
ENTRYPOINT ["conda", "run", "--no-capture-output", "-n", "myenv", "python","worker.py"]

Answer 1

You indicate in the comments that the two images are identical , aside from the final command. You can override the image's ENTRYPOINT when you run the container using the Compose entrypoint: directive:

version: '3.8'
services:
  worker:
    build: .
  web:
    build: .
    entrypoint: conda run --no-capture-output -n myenv python web.py

We can do better than this, though. An image can have both an ENTRYPOINT and a CMD , and if both are present, Docker combines them into a single command string . A typical setup is to set CMD to a specific command you want to run, and then ENTRYPOINT to some sort of wrapper that accepts that command as additional arguments. In your case, that's exactly the syntax that conda run will accept, so you can say

ENTRYPOINT ["conda", "run", "--no-capture-output", "-n", "myenv"]
CMD ["python", "worker.py"]

This can be improved even more by making sure the script is executable

# (run on the host and commit to source control)
chmod +x worker.py

and that it begins with a "shebang" line

#!/usr/bin/env python3
# (must be the very very first line, absolutely nothing before it)

Now you can directly run ./worker.py without naming the python interpreter, or set it in your Dockerfile

ENTRYPOINT ["conda", "run", "--no-capture-output", "-n", "myenv"]
CMD ["./worker.py"]

This is relevant because you can separately override entrypoint: and command: in docker-compose.yml . Overriding command: replaces the Dockerfile CMD , but leaves the ENTRYPOINT in place.

version: '3.8'
services:
  worker:
    build: .
  web:
    build: .
    command: ./web.py

Technically this builds the image twice, and if you look at docker images output, you will see both "worker" and "web" images. However, these images will be identical, even having the same image ID. The second build should run entirely from the Docker build cache and be extremely quick.

(In the question you mention an out-of-memory issue. If, as you say in comments, the two images are identical up to the final ENTRYPOINT line, then the second image build should come from the Docker cache and shouldn't require significant memory. The approach I describe here, with the same build: for both container images, could potentially run into the same issue.)