[英]How to upload url to s3 bucket using StringIO and put_object method with boto3
I need to upload URLs to an s3 bucket and am using boto3.我需要将 URL 上传到 s3 存储桶并且正在使用 boto3。 I thought I had a solution with this question: How to save S3 object to a file using boto3 but when I go to download the files, I'm still getting errors.我以为我有这个问题的解决方案: 如何使用 boto3 将 S3 object 保存到文件中,但是当我 go 下载文件时,我仍然遇到错误。 The goal is for them to download as audio files, not URLs.目标是让他们下载为音频文件,而不是 URL。 My code:我的代码:
for row in list_reader:
media_id = row['mediaId']
external_id = row['externalId']
with open('10-17_res1.csv', 'a') as results_file:
file_is_empty = os.stat('10-17_res1.csv').st_size == 0
results_writer = csv.writer(
results_file, delimiter = ',', quotechar = '"'
)
if file_is_empty:
results_writer.writerow(['fileURL','key', 'mediaId','externalId'])
key = 'corpora/' + external_id + '/' + external_id + '.flac'
bucketname = 'my_bucket'
media_stream = media.get_item(media_id)
stream_url = media_stream['streams'][0]['streamLocation']
fake_handle = StringIO(stream_url)
s3c.put_object(Bucket=bucketname, Key=key, Body=fake_handle.read())
My question is, what do I need to change so that the file is saved in s3 as an audio file, not a URL?我的问题是,我需要更改什么以便将文件保存在 s3 中作为音频文件,而不是 URL?
I solved this by using the smart_open module:我通过使用 smart_open 模块解决了这个问题:
with smart_open.open(stream_url, 'rb',buffering=0) as f:
s3.put_object(Bucket=bucketname, Key=key, Body=f.read())
Note that it won't work without the 'buffering=0' parameter.请注意,如果没有 'buffering=0' 参数,它将无法工作。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.