import boto3
import os
client = boto3.client('connect')
s3 = boto3.resource(
service_name='s3',
region_name='us-west-2',
aws_access_key_id=aws_access_key_id,
aws_secret_access_key=aws_secret_access_key
)
for my_bucket_object in s3.Bucket("my_bucket").objects.filter(Prefix="user/folder/"):
s3.Object(my_bucket_object.bucket_name, my_bucket_object.key).download_file(f'./aws/{my_bucket_object.key}')
However, when I iterate over multiple files, and use the key as input for download_file, I get the following error message. Target key's name seems to be changing?
FileNotFoundError: [Errno 2] No such file or directory: './aws/user/folder\.7g4DBa9A'
I have the following two questions:
=========================================================================== Found the answer thanks to Marcin's comment. After iteratively printing all the outputs, it seemed the first one was the "folder", which translated to strange names when downloading.
ie.
user/folder/
user/folder/file1
user/folder/file2
etc.
Thus, ignoring that first iteration was able to solve it.
for obj in my_bucket.objects.filter(Prefix=prefix):
output_file = obj.key.split('/')[-1]
if output_file == "":
continue
else:
s3.Object(bucket_name=my_bucket.name, key=my_bucket_object.key).download_file(arbitrary output path)
is there a way to separate file names from "subdirectories"
You can split the key by /
and take the last element before you do download_file
:
output_file = my_bucket_object.key.split('/')[-1]
s3.Object(my_bucket_object.bucket_name, my_bucket_object.key).download_file(f'./aws/output_file')
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.