How to categorize files from Amazon S3 ls command?

Question

To make a list of file names in an Amazon S3 bucket using the following (in Python):

import os
os.system('aws s3 ls s3://bucket --recursive --human-readable --summarize')

The output is:

2021-10-02 21:37:53   10 MiB img.txt
2021-10-02 21:37:53   10 MiB img.jpg
2021-10-02 21:32:57   10 MiB img.json
2021-10-02 21:32:58   10 MiB img.png

Total Objects: 4
Total Size: 40 MiB

The desirable output should be:

Total Objects: 4
Total Size: 40 MiB
.jpg count: 1
.png count 1
.json count 1
.txt count 1

and Total Size is a variable that could be used later in that program.

Answer 1

Here's an example of doing it with the boto3 library:

import boto3

extensions = {}
object_count = 0
total_size = 0

s3_resource = boto3.resource('s3')

for object in s3_resource.Bucket('my-bucket').objects.all():
    key = object.key
    object_count += 1
    total_size += object.size

    ext = key[key.rfind('.'):]
    extensions[ext] = extensions.get(ext, 0) + 1

print('Total Objects:', object_count)
print('Total size (MiB):', total_size // 1024 // 1024)
for ext in extensions:
    print(ext, extensions[ext])