使用boto3从AWS S3下载特定文件

Question

I am trying to download files from Amazon S3 to my local computer.

I have about 100,000 jpeg files in my Bucket, like SFADOK_1737372938_26465755_29573736.jpeg .

Now I want to download all files with the ending: *_29573736.jpeg

Could anyone please help me?

Answer 1

One simple way would be to use the AWS Command-Line Interface (CLI) :

aws s3 cp s3://my-bucket/ . --recursive --exclude '*' --include '*_29573736.jpeg'

If you wish to do it via a boto3 program, the program would need to:

• Obtain a listing of objects from the bucket using list_objects_v2()
• Loop through each object in the listing that is returned:
  • If the Key matches your desired pattern, then...
  • Use download_file() to download the object

With so many objects in the bucket, you will either need to loop through list_objects_v2() using the ContinuationToken or use a paginator to automatically loop through the results. This is because a maximum of 1000 objects will be returned per API call.