Creating a docker container that runs Ubuntu with Python 3.6+ and Pip

Question

I'm trying to install Python 3.6 or above with pip in an docker container that runs Ubuntu.I've tried quite a few things with no success

FROM ubuntu:18.04
RUN apt update
RUN apt install software-properties-common -y
RUN add-apt-repository ppa:deadsnakes/ppa
RUN ln -s /usr/bin/pip3 /usr/bin/pip
RUN ln -s /usr/bin/python3.8 /usr/bin/python
RUN apt install python3.8 -y
RUN apt install pip

RUN pip install auto-sklearn
RUN pip install pandas

ADD test.py /

CMD [ "python", "./test.py" ]

This returns "Unable to locate package pip." I tried removing "apt install pip" incase Python 3.8 comes with it, but it gives me the error: "pip: not found."

FROM ubuntu:18.04
RUN apt install software-properties-common -y
RUN add-apt-repository ppa:deadsnakes/ppa
RUN apt-get update
RUN install python3-pip
RUN pip install auto-sklearn
RUN pip install pandas

ADD test.py /

CMD [ "python", "./test.py" ]

This installs pip, but auto-sklearn requires Python version 3.6 or higher and this installs a lower version. Auto-sklearn requires Linux as well which is why I'm using "FROM ubuntu" rather than "FROM python" cause "FROM python" seems to build a container on whatever native OS is running on the computer building the container, which for me is Windows.

Answer 1

I see two consecutive issues here, so let's address them accordingly:

Issue 1: missing pip in the Ubuntu image

This returns "Unable to locate package pip." I tried removing "apt install pip" incase Python 3.8 comes with it, but it gives me the error: "pip: not found."

That's right. If you inspect the contents of the /usr/bin directory of the pulled image, you will notice that there is no pip or pip3 there. So RUN ln -s /usr/bin/pip3 /usr/bin/pip line in your Dockerfile does nothing. Even when python3.6 gets installed in the container (after calling apt install software-properties-common -y ), you do not get pip with it.

Solution: install pip

The following commands can be used to install python3.6 binary and the corresponding pip :

RUN apt-get update
RUN apt-get install python3-pip

This installs both python3.6 and pip3 in the /usr/bin directory of your ubuntu:18/04 container.

Issue 2: auto-sklearn requires python >= 3.7

Even if you manage to get both python3.6 and pip for python3.6 , installation of auto-sklearn might still fail with the following error:

    RuntimeError: Python version >= 3.7 required.

This is because some of the dependencies (eg ConfigSpace package) require python version >= 3.7.

Solution:

This answer explains how to install pip for python3.8 on Ubuntu: https://stackoverflow.com/a/63207387/15043192

You can follow it or get both pip and python3.8 installed using the following sequence:

Install python3.8:

RUN add-apt-repository ppa:deadsnakes/ppa
RUN apt update
RUN apt install python3.8

Install python3.6 and pip for python3.6

RUN apt install python3-pip

Now if you execute python3.6 -m pip --version in the container, you would get something like (the version of pip might be different):

pip 9.0.1 from /usr/lib/python3/dist-packages (python 3.6)

Install pip for python3.8

Note: here we use previously installed pip for python3.6 to install pip for python3.8 . Do not ask me why:-)

RUN python3.8 -m pip install pip --upgrade

Install auto-sklearn

RUN python3.8 -m pip install auto-sklearn

Note: the command above might also install pandas package among other dependencies of auto-sklearn .

Create a symbolic link to python3.8

This would change the default

RUN ln -s /usr/bin/python3.8 /usr/bin/python

Now if you execute python -m pip --version in the container, you would get something like (the version of pip might be different):

pip 21.2.4 from /usr/local/lib/python3.8/dist-packages/pip (python 3.8)

Grand finale:

In the end, your Dockerfile should look like this:

FROM ubuntu:18.04

RUN add-apt-repository ppa:deadsnakes/ppa
RUN apt update
RUN apt install python3.8
RUN apt install python3-pip
RUN python3.8 -m pip install auto-sklearn
RUN python3.8 -m pip install pandas
RUN ln -s /usr/bin/python3.8 /usr/bin/python

ADD test.py /

CMD [ "python", "./test.py" ]

NB

To avoid messing around with different versions of python and pip , you might want to have a look into virtual environments .