How to debug docker push to google cloud artifacts registry error "manifest unknown: Requested entity was not found."?

Question

We push images from gitlab-ci to google artifacts registry in order to deploy to google cloud run and it suddenly stopped working.

the script is

- echo $SERVICE_ACCOUNT_KEY > ./____keyfile.json
- gcloud auth activate-service-account --key-file="./____keyfile.json"
- gcloud auth configure-docker europe-west1-docker.pkg.dev
- docker pull $DOCKER_IMAGE:$DOCKER_IMAGE_TAG # pulls the previously build image that was pushed to gitlab's own registy
- docker tag $DOCKER_IMAGE:$DOCKER_IMAGE_TAG europe-west1-docker.pkg.dev/project-id/repository/$DOCKER_IMAGE_NAME:$DOCKER_IMAGE_TAG
- docker push europe-west1-docker.pkg.dev/project-id/repository/$DOCKER_IMAGE_NAME:$DOCKER_IMAGE_TAG

the last push shows:

The push refers to repository [europe-west1-docker.pkg.dev/project-id/repository/some-name]
bcf45157cf76: Preparing
151948c565a1: Preparing
5455690f232d: Preparing
65f565da952a: Preparing
66b19dc4e200: Preparing
e796b82d6587: Preparing
dcd9c7cf433b: Preparing
1f1a410f80c7: Preparing
b45078e74ec9: Preparing
e796b82d6587: Waiting
dcd9c7cf433b: Waiting
1f1a410f80c7: Waiting
b45078e74ec9: Waiting
151948c565a1: Layer already exists
65f565da952a: Layer already exists
bcf45157cf76: Layer already exists
66b19dc4e200: Layer already exists
5455690f232d: Layer already exists
e796b82d6587: Layer already exists
dcd9c7cf433b: Layer already exists
b45078e74ec9: Layer already exists
1f1a410f80c7: Layer already exists
manifest unknown: Requested entity was not found.

unfortunatly this error message "manifest unknown: Requested entity was not found." is very generic and does not show what went wrong. Is there a way to show more verbose logs?

Answer 1

Altough I did not manage to get more information from docker push , the error seems to have been caused by some outage on google's side in the specific region (in that case europe-west1). Switching the region resolved the issue.