在使用 Dockerfile 和 docker-compose 的 Python 部署中，用户定义模块的“没有名为 x 的模块”

Question

I'm trying to deploy a Python app as a Docker container using Dockerfile and docker-compose.

The project structure is this:

ms-request
    - src
        __init__.py
        - exceptions
            __init__.py
            ms_request_exceptions.py
        - messaging
            __init__.py
            receive_rabbit.py
            send_rabbit.py
        - request
            __init__.py
            bsrequest.py
    - test
        __init__.py
        test_bsrequest.py
    Dockerfile
    requirements.txt

In my receive_rabbit.py script, I am importing functions from the request and messaging packages like so:

from src.request import bsrequest
from src.messaging.send_rabbit import send_message

Executing this using PyCharm works fine. Running it from the command line initially didn't work, until I updated the PYTHONPATH using export PYTHONPATH=${PYTHONPATH}:. .

I would like to deploy this as a Docker container, so I created a Dockerfile and an entry in my docker-compose.yml for the project.

Dockerfile:

FROM python:3
WORKDIR /bsreq
COPY requirements.txt .
RUN pip install -r requirements.txt
COPY src/ ./src
COPY test/ ./test
RUN export PYTHONPATH=${PYTHONPATH}:.
CMD [ "python", "/bsreq/src/messaging/receive_rabbit.py" ]

docker-compose.yml:

version: "3.3"

services:
        rabbitmq: [...]
        bs-request:
                build: ./ms-request/
                depends_on:
                        - rabbitmq
                env_file:
                        - rabbit.env
        [...]

Running this using docker-compose up bs-request always ends in a crash with the error No module named 'src' .

I have tried multiple variations of inputs for the WORKDIR, COPY, PYTHONPATH and CMD lines in the Dockerfile. All lead to the same error. I've tried relative imports, which throw Attempted relative import with no known parent package .

I hope this is an issue others have encountered before. What do I need to do to get this deployment working?

Answer 1

docker layers-way building an image makes your export unusable right after the associated RUN command.

FROM python:3
WORKDIR /bsreq
COPY requirements.txt .
RUN pip install -r requirements.txt
COPY src/ ./src
COPY test/ ./test
RUN export PYTHONPATH=${PYTHONPATH}:.  #--> exporting
CMD [ "python", "/bsreq/src/messaging/receive_rabbit.py" ]  #--> last export is not persistent 

as a workaround you set environment variables that will persist through the build AND in the final image with ENV PYTHONPATH=${PYTHONPATH}:. command.
extra read: https://vsupalov.com/docker-build-time-env-values/

any how, suggested method is to write a setup.py file and install your package with python setup.py install so it would be installed as a package and imports would work.

PS a better, more updated way would be to use tools such as poetry that uses pyproject.toml (according to PEP 518, 517) which the future way of python!
bonus read: https://python-poetry.org/

good luck!