如何使用 python 从 google-cloud-platform 下载我的数据？

Question

I have my data on google cloud platform and i want to be able to be able to download it locally, this is my first time trying that and eventually i'll use the downloaded data with my python code.

I have checked the docs, like https://cloud.google.com/genomics/downloading-credentials-for-api-access and https://cloud.google.com/storage/docs/cloud-console i have successfully got the Json file for my first link, the second one is where u'm struggling, i'm using python 3.5 and assuming my json files name is data.json i have added the following code:

os.environ["file"] = "data.json"
urllib.request.urlopen('https://storage.googleapis.com/[bucket_name]/[filename]')

first of all i don't even know what i should call the value near environ so i just called it file, not sure how i'm supposed to fill it and i got access denied on the second line, obviously it's not how to download my file as there is no destination local repository or anything in that command any guidance will be appreciated.

Edit:

   from google.cloud.storage import Blob

    os.environ["GOOGLE_APPLICATION_CREDENTIALS"] = "credentials/client_secret.json"
    storage_client = storage.Client.from_service_account_json('service_account.json')
    client = storage.Client(project='my-project')
    bucket = client.get_bucket('my-bucket')
    blob = Blob('path/to/my-object', bucket)
    download_to_filename('local/path/to/my-file')

I'm getting unresolved reference for storage and download_to_filename and should i replace service_account.json with credentials/client_secret.json . Plus i tried to print the content of os.environ["GOOGLE_APPLICATION_CREDENTIALS"]['installed'] like i'd do with any Json but it just said i should give numbers meaning it read the input path as regular text only.

Answer 1

You should use the idiomatic Google Cloud library to run operations in GCS.

With the example there, and knowing that the client library will get the application default credentials, first we have to set the applicaiton default credentials with

gcloud auth application-default login 

===EDIT===

That was the old way. Now you should use the instructions in this link .

This means downloading a service account key file from the console , and setting the environment variable GOOGLE_APPLICATION_CREDENTIALS to the path of the downloaded JSON.

Also, make sure that this service account has the proper permissions on the project of the bucket.

Or you can create the client with explicit credentials. You'll need to download the key file all the same, but when creating the client, use:

storage_client = storage.Client.from_service_account_json('service_account.json')

==========

And then, following the example code:

from  google.cloud import storage

client = storage.Client(project='project-id')
bucket = client.get_bucket('bucket-id')
blob = storage.Blob('bucket/file/path', bucket)
blob.download_to_filename('/path/to/local/save')

Or, if this is a one-off download, just install the SDK and use gsutil to download:

gsutil cp gs://bucket/file .