使用 Boto3 导航到 s3 中的子文件夹，而无需访问 ObjectList

Question

I am kinda stuck right now.

Basically i am trying to retrieve files from a s3 bucket with boto3. It all used to work, but now the administrators of the s3 bucket decided to revoke permissions for parent folders, because the bucket is shared with a lot of companies whose data we shall not access (for obvious reasons).

Basically this is how we used to get to our files:

objects = s3_bucket.objects.filter(Prefix=s3_folder)

And then we just were going through those files as shown in the documentation:

for obj in objects:

But now this just throws an error:

An error occurred (AccessDenied) when calling the ListObjects operation: Access Denied

What i am unable to figure out now is how i can connect to a s3 bucket but only retrieve the object list starting from a specific path.

Let's say my files are under /all-data/reports/company-2/data/csv/year-month-day/*****.csv I know the entire path from /all-data/ to /csv/ but i need to list the contents of the csv folder (year-month-day folders). Also i only can retrieve the object list starting from /company-2.

I cannot find any function in the boto3 documentation which allows me to either directly connect to a subfolder in the bucket or otherwise let me navigate through the bucket down to /csv where i could start the list objects.

If there is any of you who could help me out i would really appreciate it, i could not find anyone else facing the problem yet :)

Answer 1

所以基本上我唯一的选择似乎是让管理员设置 IAM 规则，如下所述： https : //www.reddit.com/r/aws/comments/ofh4dp/navigate_to_subfolders_in_s3_with_boto3_without/h4ceohy?utm_source=share&utm_medium =web2x&context=3