How to get contents of an AWS S3 bucket programmatically in Python

Question

I can run command aws --profile minio s3 ls s3://aa/bb/ in a terminal successfully to get the contents of that particular bucket on minio, but when I run below code in Python, it returns an empty string.

import os
stream = os.popen('aws --profile minio s3 ls s3://aa/bb/')
stream.read()

And when I change the second line so that I query the contents of a local folder instead, like stream = os.popen('ls /Users/cc/') , the contents of that local folder are printed successfully as well.

When I execute the first command using os.system('aws --profile minio s3 ls s3://aa/bb/') , it returns 256 as the output.

So how to access the contents of a minio bucket programmatically in Python?

Answer 1

With the caveat that I haven't used minio, here's how I'd use boto3 (the AWS python sdk) in a python script to do what your CLI command does:

import boto3

session = boto3.session.Session(profile_name='minio')
client = session.client('s3')

response = client.list_objects_v2(
    Bucket='aa',
    Prefix='bb',
)

for item in response['Contents']:
    print(item['Key'])

boto3 on GitHub

boto3 docs