How to Recursively Delete S3 Objects of Size 0 using the AWS CLI

Question

Is there an easy way using the AWS CLI to delete all size 0 objects under a prefix?

For example if our s3 prefix looks like this:

$ aws s3 ls --recursive s3://bucket/prefix
2022-04-20 10:39:51          0 empty_file
2022-04-20 10:39:52         21 top_level_file
2022-04-14 15:01:34          0 folder_a/nested_empty_file
2022-04-23 03:35:02         42 folder_a/dont_delete_me

I would like an aws cli command line invocation to just delete empty_file and folder_a/nested_empty_file .

I know this could be done via boto or any other number of s3 api implementations, but it feels like I should be able to do this in a one-liner from the command line given how simple it is.

Answer 1

Using the aws s3api subcommand and jq for json wrangling we can do the following:

aws s3api delete-objects --bucket bucket --delete "$(aws s3api list-objects-v2 --bucket bucket --prefix prefix --query 'Contents[?Size==`0`]' | jq -c -r '{ "Objects": [.[] | {"Key": .Key}] }')"

aws s3api does not yet support reading from stdin (see GH PR here: https://github.com/aws/aws-cli/pull/3209 ) so we need to pass the list of objects via sub-shell expansion, which is unfortunately a little awkward but still meets my requirements of being a one-liner.