将文件从 Docker 容器上传到 Google Cloud Storage

Question

I'm putting together a CI/CD pipeline on GKE based on this guide by Google.

Everything is working well, except that as it's a Django project I also need to run the collectstatic command and upload the files to Google Storage.

In the Dockerfile I have the following commands:

RUN python manage.py collectstatic --noinput
RUN gsutil rsync -R /home/vmagent/app/myapp/static gs://mystorage/static

Collectstatic works as expected, but the gsutil upload fails with the following error message:

ServiceException: 401 Anonymous caller does not have storage.objects.create access to mystorage/static/...

What's the best way to authenticate gsutil?

Answer 1

If you are running your docker container on GCP (GKE), you use application credentials to authenticate a pod, which then has the same permissions as that service account. More information about this can be found here . Both GKE and other kubernetes clusters allow you to import key files as secrets. On all Kubernetes platforms this is done with the following commands. The full guide can be found here .

kubectl create secret generic pubsub-key --from-file=key.json=PATH-TO-KEY-FILE.json

Then set the environment variable in your manifest like this:

 env:
    - name: GOOGLE_APPLICATION_CREDENTIALS
      value: /var/secrets/google/key.json

Too bad running pods under a certain role is not yet possible. GCP operates with service accounts, not with roles like AWS. In AWS you can assign a role to a task, which grants a container permissions under that role.