C# : How to AWS S3 Bucket Size

Question

I want to get bucket size of AWS S3 by using .net.

Want to calculated allocated space, used space for the bucket.

1. If my bucket name is XYZ.
2. Inside my bucket there are n numbers of subfolders.
3. I want complete size of XYZ bucket.

for this access your have to give permission to bucket for public access through S3 browser or AWS console. Will get output like below

<Key>File Name</Key>
<LastModified>Last Modified Date</LastModified>
<ETag>Some Text</ETag>
<Size>173266</Size>

Size : in Bytes

Now I have calculate one by one file size and then to do calculations.

If any one can suggest good idea to get complete bucket size in one method that would be very good for me.

Answer 1

There is no API call available to obtain the size of an Amazon S3 bucket.

You can either:

• Call ListObjects() to obtain a listing of all objects and total their sizes (note: Only 1000 objects are returned per call, so it would need to loop until all objects are returned), or
• Use Amazon S3 Inventory , which can provide a daily or weekly CSV file listing all objects. You can then calculate the total bucket size from that file.

Another option (I haven't tried it) might be to use Amazon S3 Storage Lens , which can track metrics in S3.

See also:

• AWS S3: how do I see how much disk space is using
• How do I find the total size of my AWS S3 storage bucket or folder?

Answer 2

Cloud storage has no folders. Whether it's AWS S3 or Azure Blob storage, a bucket is a flat list of files. Folders are emulated by treating a character in the name as a separator. A bucket isn't a disk either, so it has no allocated vs used space statistics. Objects are stored in blob storage arrays that use compression and deduplication.

If you want to find out what storage you're actually billed for, it's the total object size, not the actual physical storage. If you upload the same 1MB file 100 times you'll be billed for 100MB even if just 1MB are actually stored.

Calculating the total size of a bucket is too expensive, which is why it's not updated automatically. If calculating the size of a local folder is slow, imagine how much slower it is to calculate the size of a bucket with 1M objects that are actively being updated.

The only way to get the used space is to list the objects and sum their sizes. This operation is billed, no matter what tool is used to perform it.

The article Find out the size of your Amazon S3 buckets shows which tools and services can be used to monitor the actual size.

• Amazon S3 console
• S3 Storage Lens
• Amazon CloudWatch
• Amazon S3 Inventory

These services calculate and cache the size periodically, to avoid delays and extra billing.

The other option is to list and sum all objects either through the AWS CLI or code. You can use the code in Listing object keys programmatically to list all objects in a bucket one page at a time and sum the sizes:

public static async Task<long> GetBucketSizeAsync(IAmazonS3 client, string bucketName)
{
    long total=0;
    var request = new ListObjectsV2Request
    {
        BucketName = bucketName,
    };

    var response = new ListObjectsV2Response();

    do
    {
        response = await client.ListObjectsV2Async(request);

        total += response.S3Objects.Sum(obj =>obj.Size);

        request.ContinuationToken = response.NextContinuationToken;
    }
    while (response.IsTruncated);

    return total;
}

Remember, ListObjectsV2Async is billed. You should cache the results instead of calling this all the time.