如何在 S3 boto3 中刪除多個文件和特定模式

Question

Python可以刪除S3中的特定多個文件嗎？

我想刪除具有特定擴展名的多個文件。

此腳本刪除所有文件。

這些是我要刪除的各種特定文件：

XXX.tar.gz
XXX.txt

** 當前代碼：**（所有文件已刪除）

import boto3

accesskey = "123"
secretkey = "123"
region = "ap-northeast-1"

s3 = boto3.resource ('s3', aws_access_key_id = accesskey, aws_secret_access_key = secretkey, region_name = region)

bucket = s3.Bucket ('test')
files = [os.key for os in bucket.objects.filter (Prefix = "myfolder / test /")]
tar_files = [file to file in files if file.endswith ('tar.gz')]

#print (f'All files: {files} ')
#print (f'CSV files: {csv_files} ')

objects_to_delete = s3.meta.client.list_objects (Bucket = "test", Prefix = "myfolder / test /")

delete_keys = {'Objects': []}
delete_keys ['Objects'] = [{'Key': tar_files} for tar_files in [obj ['Key'] for obj in objects_to_delete.get ('Content', [])]]

s3.meta.client.delete_objects (Bucket = "test", Delete = delete_keys)

如果有人知道，請告訴我。

Answer 1

假設您要從給定的存儲桶和前綴中刪除*.tar.gz和*.txt文件，這將起作用：

import boto3

s3_resource = boto3.resource('s3')

bucket = s3_resource.Bucket('my-bucket')
objects = bucket.objects.filter(Prefix = 'myfolder/')

objects_to_delete = [{'Key': o.key} for o in objects if o.key.endswith('.tar.gz') or o.key.endswith('.txt')]

if len(objects_to_delete):
    s3_resource.meta.client.delete_objects(Bucket='my-bucket', Delete={'Objects': objects_to_delete})

Answer 2

1. 迭代您的 S3 存儲桶
2. 對於每個存儲桶，遍歷文件
3. 刪除請求的文件類型

import boto3
s3 = boto3.resource('s3')
   
   for bucket in s3.meta.client.list_buckets()['Buckets']:
       for count, obj in enumerate(s3.Bucket(bucket['Name']).objects.filter()):
           if obj.key.endswith('.tar.gz') or obj.key.endswith('.txt'):
               print("{}: deleting: {} from: {}".format(count, obj.key, bucket['Name']))
               s3.meta.client.delete_object(Bucket=bucket['Name'], Key=obj.key)

我使用這種方法是因為，對於大型存儲桶，創建完整列表然后刪除它可能需要一些時間。 這樣，您就可以看到進度。