Docker Buildx feature - does not cache

Question

I'm trying to build a multi-arch docker image using the buildx experimental feature. The problem I face is that whenever I do a "make" or a "make install" docker does not cache that layer.

I'm using different cmake commands depending on the arch.

What am I missing? Is what I am trying to achieve posible or should I just make multiple dockerfiles?

FROM ubuntu:18.04 as builder

#RUN add-apt-repository "deb http://security.ubuntu.com/ubuntu xenial-security main"

RUN set -ex && \
    apt update && apt upgrade -y && \
    apt install -y build-essential cmake pkg-config && \
    apt install -y libjpeg-dev libtiff5-dev libpng-dev && \
    apt install -y libavcodec-dev libavformat-dev libswscale-dev libv4l-dev && \
    apt install -y libxvidcore-dev libx264-dev && \
    apt install -y libfontconfig1-dev libcairo2-dev && \
    apt install -y libgdk-pixbuf2.0-dev libpango1.0-dev && \
    apt install -y libgtk2.0-dev libgtk-3-dev && \
    apt install -y libatlas-base-dev gfortran && \
    apt install -y libhdf5-dev libhdf5-serial-dev && \
    apt install -y python3.7 python3.7-dev && \
    apt install -y wget unzip git

RUN wget https://bootstrap.pypa.io/get-pip.py
RUN python get-pip.py && python3 get-pip.py
RUN rm -rf ~/.cache/pip

RUN python3 -m pip install --upgrade setuptools

RUN python3.7 get-pip.py
RUN python3.7 -m pip install --upgrade setuptools

RUN python3.7 --version
RUN python3 --version

#RUN git clone https://github.com/jasperproject/jasper-client.git jasper && \
#       chmod +x jasper/jasper.py && \
#       python3.7 -m pip install -r jasper/client/requirements.txt
#
#RUN apt install -y ash

RUN ARCH=`uname -m` && echo $ARCH

RUN /bin/bash -c 'set -xe && ARCH=`uname -m` && \
    if [ "$ARCH" == "x86_64" ]; then \
       echo "x86_64" && \
       apt-get install -y software-properties-common && \
       add-apt-repository "deb http://security.ubuntu.com/ubuntu xenial-security main" && \
       apt update && \
       apt install -y libjasper-dev ; \
    fi'

RUN /bin/bash -c 'set -xe && ARCH=`uname -m` && \
    if [[ "$ARCH" == arm* ]]; then \
       echo "probably arm $ARCH" && \
       git clone https://github.com/jasperproject/jasper-client.git jasper && \
        chmod +x jasper/jasper.py && \
        python3.7 -m pip install -r jasper/client/requirements.txt ; \
    fi'

ARG NR_PROCS=1

ENV BUILD_FOLDER=/tmp_build/

RUN mkdir ${BUILD_FOLDER}

WORKDIR ${BUILD_FOLDER}


ADD https://github.com/opencv/opencv/archive/4.1.1.zip opencv.zip
ADD https://github.com/opencv/opencv_contrib/archive/4.1.1.zip opencv_contrib.zip


RUN set -ex && \
    unzip -qq opencv.zip && \
    unzip -qq opencv_contrib.zip && \
    mv opencv-4.1.1 opencv && \
    mv opencv_contrib-4.1.1 opencv_contrib

RUN set -ex && mkdir opencv/build && \
    echo "unzipped! ";

WORKDIR  ${BUILD_FOLDER}opencv/build

RUN python3.7 -m pip install numpy


RUN /bin/bash -c 'set -xe && ARCH=`uname -m` && \
    if [ "$ARCH" == "x86_64" ]; then \
       echo "x86_64" && \
       cmake -D CMAKE_BUILD_TYPE=RELEASE \
          -D BUILD_opencv_python3=yes \
          -D CMAKE_INSTALL_PREFIX=/usr/local \
          -D OPENCV_ENABLE_NONFREE=ON \
          -D CMAKE_SHARED_LINKER_FLAGS=-latomic \
          -D OPENCV_EXTRA_MODULES_PATH=${BUILD_FOLDER}opencv_contrib/modules \
          -D INSTALL_C_EXAMPLES=OFF \
          -D INSTALL_PYTHON_EXAMPLES=OFF \
          -D BUILD_EXAMPLES=OFF \
          -D PYTHON3_EXECUTABLE=$(which python3.7) \
          -D PYTHON_INCLUDE_DIR=$(python3.7 -c "from distutils.sysconfig import get_python_inc; print(get_python_inc())") \
          -D PYTHON_INCLUDE_DIR2=$(python3.7 -c "from os.path import dirname; from distutils.sysconfig import get_config_h_filename; print(dirname(get_config_h_filename()))") \
          -D PYTHON_LIBRARY=$(python3.7 -c "from distutils.sysconfig import get_config_var;from os.path import dirname,join ; print(join(dirname(get_config_var('LIBPC')),get_config_var('LDLIBRARY')))") \
          -D PYTHON3_NUMPY_INCLUDE_DIRS=$(python3.7 -c "import numpy; print(numpy.get_include())") \
          -D PYTHON3_PACKAGES_PATH=$(python3.7 -c "from distutils.sysconfig import get_python_lib; print(get_python_lib())") \
          .. ; \
    fi'

RUN /bin/bash -c 'set -xe && ARCH=`uname -m` && \
    if [[ "$ARCH" == arm* ]]; then \
       echo "probably arm  $ARCH" && \
         cmake -D CMAKE_BUILD_TYPE=RELEASE \
          -D BUILD_opencv_python3=yes \
          -D CMAKE_INSTALL_PREFIX=/usr/local \
          -D OPENCV_ENABLE_NONFREE=ON \
          -D ENABLE_NEON=ON \
          -D ENABLE_VFPV3=ON \
          -D CMAKE_SHARED_LINKER_FLAGS=-latomic \
          -D OPENCV_EXTRA_MODULES_PATH=${BUILD_FOLDER}opencv_contrib/modules \
          -D INSTALL_C_EXAMPLES=OFF \
          -D INSTALL_PYTHON_EXAMPLES=OFF \
          -D BUILD_EXAMPLES=OFF \
          -D PYTHON3_EXECUTABLE=$(which python3.7) \
          -D PYTHON_INCLUDE_DIR=$(python3.7 -c "from distutils.sysconfig import get_python_inc; print(get_python_inc())") \
          -D PYTHON_INCLUDE_DIR2=$(python3.7 -c "from os.path import dirname; from distutils.sysconfig import get_config_h_filename; print(dirname(get_config_h_filename()))") \
          -D PYTHON_LIBRARY=$(python3.7 -c "from distutils.sysconfig import get_config_var;from os.path import dirname,join ; print(join(dirname(get_config_var('LIBPC')),get_config_var('LDLIBRARY')))") \
          -D PYTHON3_NUMPY_INCLUDE_DIRS=$(python3.7 -c "import numpy; print(numpy.get_include())") \
          -D PYTHON3_PACKAGES_PATH=$(python3.7 -c "from distutils.sysconfig import get_python_lib; print(get_python_lib())") \
          .. ; \
    fi'

RUN set -xe && make -j${NR_PROCS} && make install && ldconfig

RUN python3.7 -c "import cv2; print(cv2.__version__)"

RUN apt clean

RUN rm -rf /var/lib/apt/lists/*

Answer 1

Remove as builder . It is used in multi-stage dockerfile.

Answer 2

There seems to be a bug, most probably related to moby/buildkit.

For more info an updates please check https://github.com/docker/buildx/issues/206