无法在 docker mysql 容器中安装 mysqlclient

Question

I am having some issue with installing mysqlclient in a Docker conatiner hat was made by a mysql optimized Docker image ( mysql/mysql-server ). The container runs on an Oracle Linux Sysrem (CentOS).

The error looks like this

Installing collected packages: mysqlclient, raven, redis, xmltodict, responses
  Running setup.py install for mysqlclient ... error
    Complete output from command /tracking_api/tracking/bin/python3.6 -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-z_jafqw3/mysqlclient/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-record-vqelrf7r/install-record.txt --single-version-externally-managed --compile --install-headers /tracking_api/tracking/include/site/python3.6/mysqlclient:
    running install
    running build
    running build_py
    creating build
    creating build/lib.linux-x86_64-3.6
    copying _mysql_exceptions.py -> build/lib.linux-x86_64-3.6
    creating build/lib.linux-x86_64-3.6/MySQLdb
    copying MySQLdb/__init__.py -> build/lib.linux-x86_64-3.6/MySQLdb
    copying MySQLdb/compat.py -> build/lib.linux-x86_64-3.6/MySQLdb
    copying MySQLdb/connections.py -> build/lib.linux-x86_64-3.6/MySQLdb
    copying MySQLdb/converters.py -> build/lib.linux-x86_64-3.6/MySQLdb
    copying MySQLdb/cursors.py -> build/lib.linux-x86_64-3.6/MySQLdb
    copying MySQLdb/release.py -> build/lib.linux-x86_64-3.6/MySQLdb
    copying MySQLdb/times.py -> build/lib.linux-x86_64-3.6/MySQLdb
    creating build/lib.linux-x86_64-3.6/MySQLdb/constants
    copying MySQLdb/constants/__init__.py -> build/lib.linux-x86_64-3.6/MySQLdb/constants
    copying MySQLdb/constants/CLIENT.py -> build/lib.linux-x86_64-3.6/MySQLdb/constants
    copying MySQLdb/constants/CR.py -> build/lib.linux-x86_64-3.6/MySQLdb/constants
    copying MySQLdb/constants/ER.py -> build/lib.linux-x86_64-3.6/MySQLdb/constants
    copying MySQLdb/constants/FIELD_TYPE.py -> build/lib.linux-x86_64-3.6/MySQLdb/constants
    copying MySQLdb/constants/FLAG.py -> build/lib.linux-x86_64-3.6/MySQLdb/constants
    copying MySQLdb/constants/REFRESH.py -> build/lib.linux-x86_64-3.6/MySQLdb/constants
    running build_ext
    building '_mysql' extension
    creating build/temp.linux-x86_64-3.6
    gcc -pthread -Wno-unused-result -Wsign-compare -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -Dversion_info=(1,3,12,'final',0) -D__version__=1.3.12 -I/usr/include/mysql -I/tracking_api/tracking/include -I/usr/include/python3.6m -c _mysql.c -o build/temp.linux-x86_64-3.6/_mysql.o -m64
    unable to execute 'gcc': No such file or directory
    error: command 'gcc' failed with exit status 1

    ----------------------------------------
Command "/tracking_api/tracking/bin/python3.6 -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-z_jafqw3/mysqlclient/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-record-vqelrf7r/install-record.txt --single-version-externally-managed --compile --install-headers /tracking_api/tracking/include/site/python3.6/mysqlclient" failed with error code 1 in /tmp/pip-install-z_jafqw3/mysqlclient/

I noticed that the issue was caused because there was no gcc in the container. After installing gcc with yum install gcc , I received another error that looks like this

Installing collected packages: mysqlclient, raven, redis, xmltodict, responses
  Running setup.py install for mysqlclient ... error
    Complete output from command /tracking_api/tracking/bin/python3.6 -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-90ek02_q/mysqlclient/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-record-rdg1c3go/install-record.txt --single-version-externally-managed --compile --install-headers /tracking_api/tracking/include/site/python3.6/mysqlclient:
    running install
    running build
    running build_py
    creating build
    creating build/lib.linux-x86_64-3.6
    copying _mysql_exceptions.py -> build/lib.linux-x86_64-3.6
    creating build/lib.linux-x86_64-3.6/MySQLdb
    copying MySQLdb/__init__.py -> build/lib.linux-x86_64-3.6/MySQLdb
    copying MySQLdb/compat.py -> build/lib.linux-x86_64-3.6/MySQLdb
    copying MySQLdb/connections.py -> build/lib.linux-x86_64-3.6/MySQLdb
    copying MySQLdb/converters.py -> build/lib.linux-x86_64-3.6/MySQLdb
    copying MySQLdb/cursors.py -> build/lib.linux-x86_64-3.6/MySQLdb
    copying MySQLdb/release.py -> build/lib.linux-x86_64-3.6/MySQLdb
    copying MySQLdb/times.py -> build/lib.linux-x86_64-3.6/MySQLdb
    creating build/lib.linux-x86_64-3.6/MySQLdb/constants
    copying MySQLdb/constants/__init__.py -> build/lib.linux-x86_64-3.6/MySQLdb/constants
    copying MySQLdb/constants/CLIENT.py -> build/lib.linux-x86_64-3.6/MySQLdb/constants
    copying MySQLdb/constants/CR.py -> build/lib.linux-x86_64-3.6/MySQLdb/constants
    copying MySQLdb/constants/ER.py -> build/lib.linux-x86_64-3.6/MySQLdb/constants
    copying MySQLdb/constants/FIELD_TYPE.py -> build/lib.linux-x86_64-3.6/MySQLdb/constants
    copying MySQLdb/constants/FLAG.py -> build/lib.linux-x86_64-3.6/MySQLdb/constants
    copying MySQLdb/constants/REFRESH.py -> build/lib.linux-x86_64-3.6/MySQLdb/constants
    running build_ext
    building '_mysql' extension
    creating build/temp.linux-x86_64-3.6
    gcc -pthread -Wno-unused-result -Wsign-compare -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -Dversion_info=(1,3,12,'final',0) -D__version__=1.3.12 -I/usr/include/mysql -I/tracking_api/tracking/include -I/usr/include/python3.6m -c _mysql.c -o build/temp.linux-x86_64-3.6/_mysql.o -m64
    _mysql.c:29:19: fatal error: mysql.h: No such file or directory
     #include "mysql.h"
                       ^
    compilation terminated.
    error: command 'gcc' failed with exit status 1

    ----------------------------------------
Command "/tracking_api/tracking/bin/python3.6 -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-90ek02_q/mysqlclient/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-record-rdg1c3go/install-record.txt --single-version-externally-managed --compile --install-headers /tracking_api/tracking/include/site/python3.6/mysqlclient" failed with error code 1 in /tmp/pip-install-90ek02_q/mysqlclient/

I realised that this was caused by the fact that mysql.h does not exist. I found in this question , that I can check weather the file exists. However, it seems that the mysql folder does not exists in the /use/local/ directory.

And this is what caused m confusion. Does anybody have any idea why did this happened? Why isn't there a mysql.h file when the docker image itself is mysql optimized? Thanks in advance.

Answer 1

You need to install mysql-community-devel , the mysql.h is in this package.

Note: you need first to uninstall mysql-community-server-minimal , and reinstall it after mysqlclient install done. Otherwise, you will encountered following conflict:

Transaction check error: file /usr/bin/mysql_config from install of mysql-community-devel-8.0.13-1.el7.x86_64 conflicts with file from package mysql-community-server-minimal-8.0.13-1.el7.x86_64

Steps as follows: (omit pip install)

yum remove -y mysql-community-server-minimal
yum install -y gcc
yum install -y mysql-community-devel
pip install mysqlclient
yum remove -y mysql-community-devel
yum install -y mysql-community-server-minimal

The output of mysqlclient install:

# pip install mysqlclient
Collecting mysqlclient
  Using cached https://files.pythonhosted.org/packages/ec/fd/83329b9d3e14f7344d1cb31f128e6dbba70c5975c9e57896815dbb1988ad/mysqlclient-1.3.13.tar.gz
Installing collected packages: mysqlclient
  Running setup.py install for mysqlclient ... done
Successfully installed mysqlclient-1.3.13
You are using pip version 8.1.2, however version 18.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.

Finally, your last question:

Why isn't there a mysql.h file when the docker image itself is mysql optimized

From the dockerhub, you can find the link of mysql/mysqlserver dockerfile, it is in here :

FROM oraclelinux:7-slim

ARG MYSQL_SERVER_PACKAGE=mysql-community-server-minimal-8.0.13
ARG MYSQL_SHELL_PACKAGE=mysql-shell-8.0.13

# Install server
RUN yum install -y https://repo.mysql.com/mysql-community-minimal-release-el7.rpm \
      https://repo.mysql.com/mysql-community-release-el7.rpm \
  && yum-config-manager --enable mysql80-server-minimal \
  && yum install -y \
      $MYSQL_SERVER_PACKAGE \
      $MYSQL_SHELL_PACKAGE \
      libpwquality \
  && yum clean all \
  && mkdir /docker-entrypoint-initdb.d

VOLUME /var/lib/mysql

COPY docker-entrypoint.sh /entrypoint.sh
COPY healthcheck.sh /healthcheck.sh
ENTRYPOINT ["/entrypoint.sh"]
HEALTHCHECK CMD /healthcheck.sh
EXPOSE 3306 33060
CMD ["mysqld"]

From above, you can see it just use the pre-built mysql server package , did not compile the source code of mysql server, so it no needs for mysql.h , which is only used when do source build.

But when you install mysqlclient use pip, as mysqlclient python package has C extension dependency, so it will have to use gcc to build, then it needs mysql.h , this made you in trouble. After install mysql.h , everything is ok.

Answer 2

You can simply do

    yum install -y --downloadonly --downloaddir=manual-install mysql-devel  && \
    rpm -i --force manual-install/*.rpm

this downloads the mysql-devel packages without installing, then installs using using directly the rpm commnad, that allows to force install

Wonder why yum nor microdnf have a --force param