[英]Delete all keys in a bucket of s3 which are three days old using python
How do I delete all the keys in an S3 bucket which are three days old using python?如何使用python删除三天前的S3存储桶中的所有键?
S3 bucket contents are like: S3 存储桶内容如下:
mybucket001/backup/1566394660_21_08_2019_backup
mybucket001/backup/1566394660_20_08_2019_backup
mybucket001/backup/1566394660_19_08_2019_backup
mybucket001/backup/1566394660_18_08_2019_backup
I need to keep only the last two days of data.我只需要保留最近两天的数据。
Here is what I tried:这是我尝试过的:
import boto
from boto.s3.key import Key
keyId='***'
sKeyId='***'
srcFileName="file name" #Name of the file to be deleted
bucketName="bucket name" #Name of the bucket, where the file resides
conn = boto.connect_s3(keyId,sKeyId) #Connect to S3
bucket = conn.get_bucket(bucketName) #Get the bucket object
k = Key(bucket,srcFileName) #Get the key of the given object
k.delete()
您可以简单地将Amazon S3 对象生命周期管理配置为在 3 天后删除对象,而不是通过代码执行此操作。
This seems like a slightly odd way to use boto
and S3;这似乎是使用boto
和 S3 的一种有点奇怪的方式; I would set up an S3 client:我会设置一个 S3 客户端:
import boto3
import datetime
s3 = boto3.client('s3')
then use the boto API to list the files in the bucket (assuming the bucket exists):然后使用boto API 列出存储桶中的文件(假设存储桶存在):
files = s3.list_objects_v2(Bucket='my-bucket')['Contents']
which will give you a list of dictionaries, each corresponding to one object/file.这将为您提供一个字典列表,每个字典对应一个对象/文件。
You can then filter this list by modified date and extract the keys:然后,您可以按修改日期过滤此列表并提取密钥:
old_files = [{'Key': file['Key']} for file in files if file['LastModified'] < datetime.now() - timedelta(days=2)]
and then again use the API to delete a certain portion of those files:然后再次使用 API删除这些文件的特定部分:
s3.delete_objects(Bucket='my-bucket', Delete={'Objects': old_files})
Try something like this:尝试这样的事情:
import boto
from datetime import datetime
from boto.s3.key import Key
keyId='***'
sKeyId='***'
srcFileName="file name" #Name of the file to be deleted
bucketName="bucket name" #Name of the bucket, where the file resides
conn = boto.connect_s3(keyId,sKeyId) #Connect to S3
bucket = conn.get_bucket(bucketName) #Get the bucket object
k = Key(bucket,srcFileName) #Get the key of the given object
today = datetime.now().day # Get current day
if today - int(k.split('_')[1]) >= 2: # check the day difference for the last 2 days
k.delete()
run for every key为每个键运行
You don't need to worry at all about the deletion as S3 itself will take that headache for you.您完全不需要担心删除,因为 S3 本身会让您头疼。
Ref https://aws.amazon.com/.../aws/amazon-s3-object-expiration/参考https://aws.amazon.com/.../aws/amazon-s3-object-expiration/
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.