Unauthorized error in GAE SDK, but it works once deployed

Question

I am running this code in a small example:

from google.cloud import storage
from google.appengine.api import app_identity

class TestB(base_handler.TRNHandler):
    #...
    def post(self):
        client = storage.Client()
        bucket_name = os.environ.get('BUCKET_NAME',
                           app_identity.get_default_gcs_bucket_name())
        bucket = client.get_bucket(bucket_name)
        #...

If I deploy this code everything works as expected. But when I run it locally (SDK), I get an error: Unauthorized: 401 Invalid Credentials. What's happening and how can I fix it?

Answer 1

I've got a pretty strong guess, although I can't be sure without seeing your exact logs and whatnot.

The google.cloud library is smart about authorization. It uses a thing called "application default credentials." If you run that code on App Engine or on a GCE instance, the code will be able to figure out which service account is associated with that instance and authorize itself with the credentials of that account.

However, when you run the program locally, the library has no way of knowing which credentials to use, and so it just makes calls anonymously. Your bucket probably hasn't granted anonymous users access (which is good), and so the call fails with a 401.

You can, however, register credentials locally with the gcloud command:

$> gcloud auth application-default login

Run that, and the library will use whatever credentials you've used to log in for a while. Alternatively, you could also make sure that the environment variable GOOGLE_APPLICATION_CREDENTIALS points to a service account's JSON key file.

There's a bunch of documentation on exactly how Application Default Credentials pick a credential.

Alternately, if you'd prefer to specify auth right in the program, you can do that too :

storage = Storage.from_service_account_json('/path/to/key_file.json')