使用boto3在桶中生成最新object的预签名s3 URL

Question

I have a s3 bucket with multiple folders. How can I generate s3 presigned URL for a latest object using python boto3 in aws for each folder asked by a user?

Answer 1

If it's a small bucket then recursively list the bucket, with prefix as needed. Sort the results by timestamp, and create the pre-signed URL for the latest.

If it's a very large bucket, this will be very inefficient and you should consider other ways to store the key of the latest file. For example: trigger a Lambda function whenever an object is uploaded and write that object's key into a LATEST item in DynamoDB (or other persistent store).

Answer 2

You can do something like

import boto3
from botocore.client import Config
import requests
bucket = 'bucket-name'
folder = '/' #you can add folder path here don't forget '/' at last

s3 = boto3.client('s3',config=Config(signature_version='s3v4'))

objs = s3.list_objects(Bucket=bucket, Prefix=folder)['Contents']     
latest = max(objs, key=lambda x: x['LastModified'])
print(latest)

print (" Generating pre-signed url...")
url = s3.generate_presigned_url(
    ClientMethod='get_object',
    Params={
        'Bucket': bucket,
        'Key': latest['Key']
    }
)
print(url)
response = requests.get(url)
print(response.url)

here it will give the latest last modified file from the whole bucket however you can update login and update prefix value as per need.

if you are using Kubernetes POD, VM, or anything you can pass environment variables or use the python dict to store the latest key if required.