python slim or alpine docker images fail to install packages using pipenv

Question

Im trying to build a docker container for my python applications however im having an issue when it comes to installing python packages via pipenv.

If i set the base image to python:3.8.5 it works however if i set the -slim version or -alpine versions they fail with the following error:

 Traceback (most recent call last):   File "/usr/local/bin/pipenv",
 line 8, in <module>
     sys.exit(cli())   File "/usr/local/lib/python3.8/site-packages/pipenv/vendor/click/core.py",
 line 829, in __call__
     return self.main(*args, **kwargs)   File "/usr/local/lib/python3.8/site-packages/pipenv/vendor/click/core.py",
 line 782, in main
     rv = self.invoke(ctx)   File "/usr/local/lib/python3.8/site-packages/pipenv/vendor/click/core.py",
 line 1259, in invoke
     return _process_result(sub_ctx.command.invoke(sub_ctx))   File "/usr/local/lib/python3.8/site-packages/pipenv/vendor/click/core.py",
 line 1066, in invoke
     return ctx.invoke(self.callback, **ctx.params)   File "/usr/local/lib/python3.8/site-packages/pipenv/vendor/click/core.py",
 line 610, in invoke
     return callback(*args, **kwargs)   File "/usr/local/lib/python3.8/site-packages/pipenv/vendor/click/decorators.py",
 line 73, in new_func
     return ctx.invoke(f, obj, *args, **kwargs)   File "/usr/local/lib/python3.8/site-packages/pipenv/vendor/click/core.py",
 line 610, in invoke
     return callback(*args, **kwargs)   File "/usr/local/lib/python3.8/site-packages/pipenv/vendor/click/decorators.py",
 line 21, in new_func
     return f(get_current_context(), *args, **kwargs)   File "/usr/local/lib/python3.8/site-packages/pipenv/cli/command.py", line
 232, in install
     retcode = do_install(   File "/usr/local/lib/python3.8/site-packages/pipenv/core.py", line 2052, in
 do_install
     do_init(   File "/usr/local/lib/python3.8/site-packages/pipenv/core.py", line 1305, in
 do_init
     do_install_dependencies(   File "/usr/local/lib/python3.8/site-packages/pipenv/core.py", line 899, in
 do_install_dependencies
     batch_install(   File "/usr/local/lib/python3.8/site-packages/pipenv/core.py", line 796, in
 batch_install
     _cleanup_procs(procs, failed_deps_queue, retry=retry)   File "/usr/local/lib/python3.8/site-packages/pipenv/core.py", line 703, in
 _cleanup_procs
     raise exceptions.InstallError(c.dep.name, extra=err_lines) pipenv.exceptions.InstallError: ERROR: Couldn't install package:
 brotlipy  Package installation failed...

Dockerfile

FROM python:3.8.5-slim

RUN pip install pipenv

WORKDIR /app

COPY ./ /app/

RUN pipenv install --sequential --deploy --verbose

RUN python -m spacy download en_core_web_sm

RUN pip install gunicorn[gevent]

EXPOSE 8080

CMD gunicorn -c src/gunicorn.conf.py  --chdir src wsgi:app

Pipfile

[[source]]
name = "pypi"
url = "https://pypi.org/simple"
verify_ssl = true

[dev-packages]
autopep8 = "*"

[packages]
pytest = "*"
aiomysql = "*"
pytest-asyncio = "*"
spacy = "*"
nltk = "*"
jsonschema = "*"
cachetools = "*"
pycountry = "*"
boto3 = "*"
aiohttp = {extras = ["speedups"], version = "*"}
install = "*"
cchardet = "*"
aiohttp-cors = "*"
pytest-aiohttp = "*"
aiohttp-devtools = "*"
json-logging = "*"

[requires]
python_version = "3.8"

Update:

After adding the package to compile i get a different error this time, it also takes a very long time to run now.

RUN apk add gcc g++ libffi-dev musl-dev \

Exception information:
Traceback (most recent call last):
  File "/root/.local/share/virtualenvs/app-4PlAip0Q/lib/python3.8/site-packages/pip/_internal/cli/base_command.py", line 216, in _main
    status = self.run(options, args)
  File "/root/.local/share/virtualenvs/app-4PlAip0Q/lib/python3.8/site-packages/pip/_internal/cli/req_command.py", line 182, in wrapper
    return func(self, options, args)
  File "/root/.local/share/virtualenvs/app-4PlAip0Q/lib/python3.8/site-packages/pip/_internal/commands/install.py", line 324, in run
    requirement_set = resolver.resolve(
  File "/root/.local/share/virtualenvs/app-4PlAip0Q/lib/python3.8/site-packages/pip/_internal/resolution/legacy/resolver.py", line 183, in resolve
    discovered_reqs.extend(self._resolve_one(requirement_set, req))
  File "/root/.local/share/virtualenvs/app-4PlAip0Q/lib/python3.8/site-packages/pip/_internal/resolution/legacy/resolver.py", line 388, in _resolve_one
    abstract_dist = self._get_abstract_dist_for(req_to_install)
  File "/root/.local/share/virtualenvs/app-4PlAip0Q/lib/python3.8/site-packages/pip/_internal/resolution/legacy/resolver.py", line 340, in _get_abstract_dist_for
    abstract_dist = self.preparer.prepare_linked_requirement(req)
  File "/root/.local/share/virtualenvs/app-4PlAip0Q/lib/python3.8/site-packages/pip/_internal/operations/prepare.py", line 482, in prepare_linked_requirement
    abstract_dist = _get_prepared_distribution(
  File "/root/.local/share/virtualenvs/app-4PlAip0Q/lib/python3.8/site-packages/pip/_internal/operations/prepare.py", line 91, in _get_prepared_distribution
    abstract_dist.prepare_distribution_metadata(finder, build_isolation)
  File "/root/.local/share/virtualenvs/app-4PlAip0Q/lib/python3.8/site-packages/pip/_internal/distributions/sdist.py", line 33, in prepare_distribution_metadata
    self.req.load_pyproject_toml()
  File "/root/.local/share/virtualenvs/app-4PlAip0Q/lib/python3.8/site-packages/pip/_internal/req/req_install.py", line 503, in load_pyproject_toml
    pyproject_toml_data = load_pyproject_toml(
  File "/root/.local/share/virtualenvs/app-4PlAip0Q/lib/python3.8/site-packages/pip/_internal/pyproject.py", line 95, in load_pyproject_toml
    raise InstallationError(
pip._internal.exceptions.InstallationError: Disabling PEP 517 processing is invalid: project specifies a build backend of setuptools.build_meta in pyproject.toml
Removed cryptography==3.0 from https://files.pythonhosted.org/packages/bf/ac/552fc8729d90393845cc3a2062facf4a89dcbe206fa78771d60ddaae7554/cryptography-3.0.tar.gz#sha256=8e924dbc025206e97756e8903039662aa58aa9ba357d8e1d8fc29e3092322053 (from -r /tmp/pipenv-9avbndah-requirements/pipenv-ituvdg4a-requirement.txt (line 1)) from build tracker '/tmp/pip-req-tracker-roedt7he'
Removed build tracker: '/tmp/pip-req-tracker-roedt7he'
Removing intermediate container de278d8d660a
 ---> bb539a91d593
Step 7/10 : RUN python -m spacy download en_core_web_sm
 ---> Running in d568290d25da
/usr/local/bin/python: No module named spacy

Answer 1

The problem can be minimized to a Dockerfile with

FROM python:3.8.5-slim
RUN pip install --no-cache-dir brotlipy

If you run this, you will get the error

...
unable to execute 'gcc': No such file or directory
error: command 'gcc' failed with exit status 1
...

The -slim and -alpine images do not include a C compiler, whereas the python:3.8.5 image does. If you install a C compiler, you will likely get other errors, like missing limit.s , ffi.h , and the C++ standard library.

Here is a minimal working example using the -alpine image.

FROM python:3.8.5-alpine
RUN apk add gcc g++ libffi-dev musl-dev \
    && python -m pip install --no-cache-dir brotlipy \
    && apk del gcc g++ libffi-dev musl-dev

Here is a complete example for the -slim image. Use pipenv run python to run the spacy part.

FROM python:3.8.5-slim

ARG DEBIAN_FRONTEND="noninteractive"

RUN pip install pipenv

WORKDIR /app

COPY ./ /app/

RUN apt-get update \
    && apt-get install --yes --no-install-recommends \
        gcc g++ libffi-dev \
    && pipenv install --sequential --deploy --verbose \
    && apt-get autoremove --yes gcc g++ libffi-dev \
    && rm -rf /var/lib/apt/lists/*

RUN pipenv run python -m spacy download en_core_web_sm

RUN pipenv install gunicorn[gevent]

EXPOSE 8080

CMD gunicorn -c src/gunicorn.conf.py  --chdir src wsgi:app