[英]Uploading Multiple Web Files to Cloud Storage With Python
I am trying to upload multiple web files to a storage bucket using python.我正在尝试使用 python 将多个 Web 文件上传到存储桶。 I have a service account set up to enable the credentials and so it should be working.
我设置了一个服务帐户来启用凭据,因此它应该可以正常工作。 However, every time I try to run this code I receive this error:
但是,每次我尝试运行此代码时,都会收到此错误:
Forbidden: 403 GET https://storage.googleapis.com/storage/v1/b/voterfile-oh?projection=noAcl : xxx@yyy.zzz.com does not have storage.buckets.get access to voterfile-oh.禁止:403 GET https://storage.googleapis.com/storage/v1/b/voterfile-oh?projection=noAcl :xxx@yyy.zzz.com 没有 storage.buckets.get 访问voterfile-oh。
Can I receive help on this issue?我可以获得有关此问题的帮助吗? I was not able to gather any helpful information from this question nor did the questioner receive an answer that resolved the issue.
我无法从这个问题中收集到任何有用的信息, 提问者也没有收到解决问题的答案。
def upload_voterfile(bucket_name):
from google.cloud import storage
from google.oauth2.service_account import Credentials
import os
import json
credentials_dict = {credentials}
with open('credentials.json', 'w') as json_file:
json.dump(credentials_dict, json_file)
credentials = Credentials.from_service_account_file('credentials.json')
client = storage.Client(credentials=credentials, project='oh-data-pipeline')
bucket = client.get_bucket(bucket_name)
county_list= ['Adams','Allen', 'Ashland', 'Ashtabula', 'Athens', 'Auglaize', 'Belmont', 'Brown', 'Butler', 'Carroll',
'Champaign', 'Clark', 'Clermont', 'Clinton', 'Columbiana', 'Coshocton', 'Crawford', 'Cuyahoga', 'Darke',
'Defiance', 'Delaware', 'Erie', 'Fairfield', 'Fayette', 'Franklin', 'Fulton', 'Gallia', 'Geauga', 'Greene',
'Guernsey', 'Hamilton', 'Hancock', 'Hardin', 'Harrison', 'Henry', 'Highland', 'Hocking', 'Holmes',
'Huron', 'Jackson', 'Jefferson', 'Knox', 'Lake', 'Lawrence', 'Licking', 'Logan', 'Lorain', 'Lucas', 'Madison',
'Mahoning', 'Marion', 'Medina', 'Meigs', 'Mercer', 'Miami', 'Monroe', 'Montgomery', 'Morgan', 'Morrow',
'Muskingum', 'Noble', 'Ottawa', 'Paulding', 'Perry', 'Pickaway', 'Pike', 'Portage', 'Preble', 'Putnam',
'Richland', 'Ross', 'Sandusky', 'Scioto', 'Seneca', 'Shelby', 'Stark', 'Summit', 'Trumbull', 'Tuscarawas',
'Union', 'Van Wert', 'Vinton', 'Warren', 'Washington', 'Wayne', 'Williams', 'Wood', 'Wyandot']
for f, g in zip(range(1, 89), county_list):
file = urllib.request.urlopen('https://www6.sos.state.oh.us/ords/f?p=VOTERFTP:DOWNLOAD::FILE:NO:2:P2_PRODUCT_NUMBER:{}'.format(f))
blob = bucket.blob('{}'.format(g))
blob.upload_from_string(data=file.read(), content_type="text/plain")
if __name__ == "__main__":
upload_voterfile(bucket_name='voterfile-oh')
I have been able to reproduce your issue, here are the steps I have followed:我已经能够重现您的问题,以下是我遵循的步骤:
gcloud iam service-accounts keys create [FILE_NAME].json --iam-account [NAME]@[PROJECT_ID].iam.gserviceaccount.com
to get account credentials file.gcloud iam service-accounts keys create [FILE_NAME].json --iam-account [NAME]@[PROJECT_ID].iam.gserviceaccount.com
以获取帐户凭据文件。 The reason behind this error is that on line 14 you're getting your bucket object through get_bucket
method.此错误背后的原因是在第 14 行,您通过
get_bucket
方法获取您的存储桶对象。 This method queries cloud Storage, requiring get permissions on your bucket but the Storage Object Creator role does not include get permissions.此方法查询云存储,需要对您的存储桶进行获取权限,但存储对象创建者角色不包括获取权限。
To solve your issue just change line 14 with this code bucket = client.bucket(bucket_name)
which directly creates a bucket object without interacting with Cloud Storage and therefore it does not raise the permission error, see corresponding reference .要解决您的问题,只需使用此代码更改第 14 行
bucket = client.bucket(bucket_name)
直接创建一个存储桶对象而不与 Cloud Storage 交互,因此它不会引发权限错误,请参阅相应的参考。
Another solution might be to change the service account role to Storage Object Admin because it includes get permissions.另一种解决方案可能是将服务帐户角色更改为 Storage Object Admin,因为它包括获取权限。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.