Google Cloud Logging Driver cannot find credentials after reboot

Question

I've followed the directions here , and everything works well until I restart my computer. After restarting, it seems like the docker daemon loses track of the Google credentials.

$ docker run --log-driver=gcplogs ...

fails with:

docker: Error response from daemon: failed to initialize logging driver: google: could not find default credentials. See https://developers.google.com/accounts/docs/application-default-credentials for more information. ERRO[0000] error waiting for container: context canceled

This is strange to me, because running $ systemctl show --property=Environment docker returns the value in my systemd configuration: Environment=GOOGLE_APPLICATION_CREDENTIALS=/etc/path/to/application_default_credentials.json

If I $ sudo systemctl restart docker , then docker runs sucessfully and logs are sent to stackdriver. But I want this docker image to run automatically on startup, and restarting docker with sudo gets in the way.

Is there a way to initialize the docker daemon with the necessary environment variables, so gcplogs is ready on boot without restarting docker?

Answer 1

I had two versions of docker installed -- one through adding docker's repo to apt, and one through snap. Running

sudo systemctl list-unit-files| grep docker | grep enabled

showed two installations of docker:

docker.service                             enabled    
snap.docker.dockerd.service                enabled

Having two docker installations was causing problems for startup. I removed the snap installation, rebooted, and everything now works.

Answer 2

I think you may try to edit the systemd: Unit dependencies and order , let docker.service start after google-accounts-daemon.service.

You can see all the service in google vm by

sudo systemctl list-unit-files| grep google | grep enabled

And you will see

google-accounts-daemon.service             enabled
google-clock-skew-daemon.service           enabled
google-instance-setup.service              enabled
google-network-daemon.service              enabled
google-shutdown-scripts.service            enabled
google-startup-scripts.service             enabled