Docker：构建 Debian 11 镜像，其中包含 Python 和 Google Cloud SDK

Question

I want ot build a Docker image with the following features:

• Must have a Debian 11 base image ( Mandatory )
• On top of it, I also need installed:
  • Python 3.
    • spaCy package (There could be others of my choice).
  • Google Cloud SDK (I will need this to run commands using gcloud and gsutil ).
    • Most likely, I will download some files stored in some Cloud Storage bucket.

I decided to use gcr.io/cloud-marketplace/google/debian11:latest because as far as I understood, that base image should have both Python and Google Cloud SDK already installed on top of it, but it did not, so I started adding what I needed on my own.

My first Dockerfile attempt (defaults) looks as follows:

FROM gcr.io/cloud-marketplace/google/debian11:latest   
RUN apt-get update -y

# Suggested by @JohnHanley, in the comments section
RUN apt-get install apt-transport-https ca-certificates gnupg -y
RUN echo "deb [signed-by=/usr/share/keyrings/cloud.google.gpg] http://packages.cloud.google.com/apt cloud-sdk main" | tee -a /etc/apt/sources.list.d/google-cloud-sdk.list && \
    curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key --keyring /usr/share/keyrings/cloud.google.gpg  add - && \
    apt-get update -y && \
    apt-get install google-cloud-cli -y

# Back to my original implementation
RUN apt-get install -y python3
RUN apt-get install -y python3-pip
RUN pip install spacy==3.2.1
CMD ["gsutil","--version"]

The error triggered:

------
 > [4/6] RUN echo "deb [signed-by=/usr/share/keyrings/cloud.google.gpg] http://packages.cloud.google.com/apt cloud-sdk main" | tee -a /etc/apt/sources.list.d/google-cloud-sdk.list &&     curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key --keyring /usr/share/keyrings/cloud.google.gpg  add - &&     apt-get update -y &&     apt-get install google-cloud-cli -y:
#6 0.403 deb [signed-by=/usr/share/keyrings/cloud.google.gpg] http://packages.cloud.google.com/apt cloud-sdk main
#6 0.403 /bin/sh: 1: curl: not found
#6 0.442 Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)).
#6 0.453 gpg: no valid OpenPGP data found.
------
executor failed running [/bin/sh -c echo "deb [signed-by=/usr/share/keyrings/cloud.google.gpg] http://packages.cloud.google.com/apt cloud-sdk main" | tee -a /etc/apt/sources.list.d/google-cloud-sdk.list &&     curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key --keyring /usr/share/keyrings/cloud.google.gpg  add - &&     apt-get update -y &&     apt-get install google-cloud-cli -y]: exit code: 2

My second Dockerfile attempt ( "If apt-key command is not supported" ) looks as follows:

FROM gcr.io/cloud-marketplace/google/debian11:latest   
RUN apt-get update -y

# Suggested by @JohnHanley, in the comments section
RUN apt-get install apt-transport-https ca-certificates gnupg -y
RUN echo "deb [signed-by=/usr/share/keyrings/cloud.google.gpg] http://packages.cloud.google.com/apt cloud-sdk main" | tee -a /etc/apt/sources.list.d/google-cloud-sdk.list && \
    curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | tee /usr/share/keyrings/cloud.google.gpg && apt-get update -y && \
    apt-get install google-cloud-sdk -y

# Back to my original implementation
RUN apt-get install -y python3
RUN apt-get install -y python3-pip
RUN pip install spacy==3.2.1
CMD ["gsutil","--version"]

The error triggered:

------
 > [4/6] RUN echo "deb [signed-by=/usr/share/keyrings/cloud.google.gpg] http://packages.cloud.google.com/apt cloud-sdk main" | tee -a /etc/apt/sources.list.d/google-cloud-sdk.list &&     curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | tee /usr/share/keyrings/cloud.google.gpg && apt-get update -y &&     apt-get install google-cloud-sdk -y:
#7 0.423 deb [signed-by=/usr/share/keyrings/cloud.google.gpg] http://packages.cloud.google.com/apt cloud-sdk main
#7 0.423 /bin/sh: 1: curl: not found
#7 0.974 Get:1 http://deb.debian.org/debian bullseye InRelease [116 kB]
#7 1.130 Get:2 http://deb.debian.org/debian bullseye-updates InRelease [44.1 kB]
#7 1.215 Get:3 http://deb.debian.org/debian-security bullseye-security InRelease [48.4 kB]
#7 1.373 Get:4 http://packages.cloud.google.com/apt cloud-sdk InRelease [6751 B]
#7 1.420 Err:4 http://packages.cloud.google.com/apt cloud-sdk InRelease
#7 1.420   The following signatures couldn't be verified because the public key is not available: NO_PUBKEY B53DC80D13EDEF05 NO_PUBKEY FEEA9169307EA071
#7 1.431 Get:5 http://deb.debian.org/debian-security bullseye-security/main amd64 Packages [262 kB]
#7 1.590 Reading package lists...
#7 1.985 W: GPG error: http://packages.cloud.google.com/apt cloud-sdk InRelease: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY B53DC80D13EDEF05 NO_PUBKEY FEEA9169307EA071
#7 1.985 E: The repository 'http://packages.cloud.google.com/apt cloud-sdk InRelease' is not signed.
------
executor failed running [/bin/sh -c echo "deb [signed-by=/usr/share/keyrings/cloud.google.gpg] http://packages.cloud.google.com/apt cloud-sdk main" | tee -a /etc/apt/sources.list.d/google-cloud-sdk.list &&     curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | tee /usr/share/keyrings/cloud.google.gpg && apt-get update -y &&     apt-get install google-cloud-sdk -y]: exit code: 100

If the section where Google Cloud SDK is installed gets commented, the rest works OK. Another important info:

• The base OS:

Edition Windows 10 Pro
Version 22H2
Installed on    ‎10/‎27/‎2022
OS build    19045.2251
Experience  Windows Feature Experience Pack 120.2212.4180.0

• Docker version: Docker version 20.10.21, build baeda1f

QUESTIONS:

1. What am I doing wrong?
2. Is there a better way to do what I want? (Image size can have up to 10GB in size, so optimization might be optional for this case).

UPDATES: This section will include lengthy answers to questions raised in the comment sections.

• 2022-12-02, @JohnHanley suggested in thew comments, to follow Google official documentation (section "Docker Tip" ) to write the Dockerfiles, which I was already doing, but did not include in the original post; I am including that now. OS and Dockerfile versions are included also.

Answer 1

You can install gcloud cli in your container with the following way:

FROM gcr.io/cloud-marketplace/google/debian11:latest   

RUN apt-get update -y
RUN apt install curl -y

ENV PATH=/google-cloud-sdk/bin:$PATH

WORKDIR /
RUN export CLOUD_SDK_VERSION="410.0.0" && \
    curl -O https://dl.google.com/dl/cloudsdk/channels/rapid/downloads/google-cloud-sdk-${CLOUD_SDK_VERSION}-linux-x86_64.tar.gz && \
    tar xzf google-cloud-sdk-${CLOUD_SDK_VERSION}-linux-x86_64.tar.gz && \
    rm google-cloud-sdk-${CLOUD_SDK_VERSION}-linux-x86_64.tar.gz && \
    ln -s /lib /lib64

RUN gcloud config set core/disable_usage_reporting true && \
    gcloud config set component_manager/disable_update_check true && \
    gcloud config set metrics/environment github_docker_images && \
    gcloud -q components install beta kubectl

ENTRYPOINT [ "sh", "-c", "gcloud --version" ]

This part is not mandatory:

RUN gcloud config set core/disable_usage_reporting true && \
    gcloud config set component_manager/disable_update_check true && \
    gcloud config set metrics/environment github_docker_images && \
    gcloud -q components install beta kubectl