[英]Load data from bucket google cloud
Here is a function to load data from google cloud bucket. 这是一个从Google Cloud Bucket加载数据的功能。
action_dataset_folder_path = 'action-data-set'
zip_path = 'actions.zip'
url='http://console.cloud.google.com/storage/browser/actions'
class LoadProgress(tqdm):
last_block = 0
def hook(self, block_num=1, block_size=1, total_size=None):
self.total = total_size
self.update((block_num - self.last_block) * block_size)
self.last_block = block_num
if not isfile(zip_path):
with LoadProgress(unit='B', unit_scale=True, miniters=1, desc='actions-Dataset') as pbar:
urlretrieve(
url,
zip_path,
pbar.hook)
if not isdir(action_dataset_folder_path):
with tarfile.open(zip_path) as tar:
tar.extractall()
tar.close()
print('All done ...!')
The file is downloaded as empty file with 73.7KB! 该文件下载为空文件,大小为73.7KB! I did not understand! 我不明白! It seems everything is good. 看来一切都很好。
Here is the code from google cloud site: python-code 这是来自谷歌云网站的代码 : python-code
from gcloud import storage
def download_blob(bucket_name, source_blob_name, destination_file_name):
"""Downloads a blob from the bucket."""
storage_client = storage.Client()
bucket = storage_client.get_bucket(bucket_name)
blob = bucket.blob(source_blob_name)
blob.download_to_filename(destination_file_name)
print('Blob {} downloaded to {}.'.format(
source_blob_name,
destination_file_name))
download_blob("datset","actions", "dataset")
You can retrieve data from Google Cloud Storage by using a GET request . 您可以使用GET请求从Google Cloud Storage检索数据。 In Python you could do this with the Requests library. 在Python中,您可以使用Requests库执行此操作。
First you need to retrieve an auth code (you can test this using OAuth 2.0 Playground ) 首先,您需要检索一个身份验证代码(可以使用OAuth 2.0 Playground进行测试)
Then you could use something like this to retrieve the data (object): 然后,您可以使用类似以下的方法来检索数据(对象):
import requests
authCode = YOUR_AUTH_CODE
auth = "Bearer " + authCode
myHeaders = {"Authorization": auth}
r = requests.get('https://www.googleapis.com/storage/v1/b/BUCKET_NAME/o/OBJECT_NAME', headers=myHeaders)
print r.text
print r.status_code
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.