Huge varience in S3 Metrics - Cloudwatch vs AWS S3 CLI vs UI 'Get Size'

Question

I have a single bucket which has very different Total Size and Number of Objects metrics between the S3 Management Console / CloudWatch, and the S3 command line tool.

S3 Management Console Metrics screen

• BucketSizeBytes: StandardStorage: 3.2TB
• NumberOfObjects: 172,000

S3 Management Console - Get Size Dialog

• Total Size: 264.0 GiB
• Total Objects: 127,030

Cloudwatch:

• BucketSizeBytes: 3.55TB
• NumberOfObjects: 172,000

Command Line Tool

aws s3 ls --summarize --human-readable --recursive s3://mybucket.com

• Total Size: 264.0 GiB
• Total Objects: 127,030

The command line tool and S3 Management Console - Get Size Dialog is returning what I'd expect.

We are being billed based on the much higher CloudWatch stats.

I wouldn't be bothered if they are close, but Cloudwatch is reporting close to 15 x as much StorageSize than the CLI tool.

There is no replication on the S3 bucket.

There are lifecycle policies to:

• Clean up incomplete multipart uploads
• Send files +1 years to Glacier
• Permenantly delete previous versions after a year

Versioning is turned on but I am pretty sure it's not used; ie files have only ever been uploaded once. I have tested a few random files to confirm, and they all only have one, the latest, version, available. I am happy to turn this off at the bucket settings, but I understand this won't 'delete' old versions (if they exist) anyway?

I don't believe it's because of Glacier stored files either, as I only recently set this lifecycle policy up, and the size mismatch was present before that.

And I'm extra confused given the S3 Management Console UI > Overview > Select all folders > Actions > Get Size reports the same as the CLI tool, 264GB.

So, seleting all filers/folders in the UI returns 264GB, but the Metrics return 3.2TB.

Can anyone please shed some light on what's going on here, and how I can clean it up! Thanks Rob

Answer 1

This was caused by 3TB of old versioned files; but the whole directory had been deleted so wasn't showing in the UI.

When I swapped the UI to show versions, the deleted folder appeared and then within that I could see all the old versions of the deleted files.

The following CLI statement helped me confirm it was caused by versioning:

aws s3api list-object-versions --bucket mybucker.com --query 'Versions[*].[Size,Size]' --output text  | awk '{s+=$1} END {printf "%.0f", s}'