How to reduce AWS S3 data transfer costs?

Question

I have a AWS S3 bucket to store a combination of public and private files for a laravel app as follows:

AWS Bucket and folders:

myapp
    |____private
    |____public

In my blade files the images are displayed as follows:

<img src="http://myapp.com/files/10"/>

This is my route:

Route::get('files/{id}', 'FileController@show');

This is my controller method to show and upload files:

public function show(Request $request, $id)
{
    $file = \App\File::findOrFail($id);

    $user = auth()->check() ? auth()->user() : new User;

    $this->authorizeForUser($user, 'show', $file);

    return Image::make(Storage::disk('s3')->get($file->path))->response();

} 


public function store(FileRequest $request)
{

    $file = $request->file('file');
    $fileType = strtolower($request->input('file_type'));
    $filePath = $fileType . '/' . Str::random(40) . '.' .  $file->getClientOriginalExtension();
   
    $user = auth()->user();

    DB::beginTransaction();

    try
    {
        $this->authorizeForUser($user, 'store', \App\File::class);

        $newFile = \App\File::create([
               "name"          => $file->getClientOriginalName(),
               "path"          => $filePath,
               "size"          => $file->getSize(),
               "mime_type"     => $file->getMimeType(),
               "type"          => $fileType, 
               "user_id"       => $user->id
        ]);

        $image = Image::make($file->getRealPath())->resize(360, 180);
        
        Storage::disk('s3')->put($filePath, $image->stream());
       
        $content = "<textarea data-type=\"application/json\">{\"ok\": true, \"message\": \"Success\", \"path\": \"" . $newFile->path . "\",  \"id\": \"" . $newFile->id . "\",  \"name\": \"" . $newFile->name . "\" }</textarea>";

        DB::commit();

    } catch (\Exception $e)
    {
        DB::rollback();

        $error =  'There was a problem uploading your file';
        $content = "<textarea data-type=\"application/json\">{\"ok\": false, \"message\": \"" . $error . "\" }</textarea>";

    }

    return response($content);

}

This is the file policy where authenticated users can access both public and private files whereas guest users can only access public files:

public function show(User $user, File $file)
{
     if ($file->type == 'public' || (!is_null($user->getKey()) && $file->type == 'private'))
          return true;
     }
}

I want to add cache support because even small number of app users produce hundreds and sometimes thouthands of GET requests on AWS S3. How can I cache files to reduce S3 costs where the bucket is private ie can only be accessed via an IAM user?

I've read some posts that recommend using cloudfront but isn't that just adding another cost and how can I implement cloudfront with the controller method I have above and where the bucket is private at root level?

Would using laravels file cache be a suitable option?

I'd be interested to know what techniques I can use with S3 and laravel, preferably with some steps and code for guidance would really help.

PS. I've already looked at the following post. Laravel AWS S3 Storage image cache

Answer 1

The easiest option is to use CloudFront and if your problem is the cost, AWS S3 doesn't charge Data Transfer when the request goes to CloudFront.

It's very to set up CloudFront with S3, and you can keep the Bucket private, you'll just need to create OAI.

Check these links below:

https://aws.amazon.com/pt/premiumsupport/knowledge-center/cloudfront-https-requests-s3/

https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/private-content-restricting-access-to-s3.html

As the CloudFront is a public service, you'll need to change your current authentication to another format (check this linkcloudfront-authorization-at-edge ) or you can serve private content using Signed URLs or Signed Cookies (check this another link PrivateContent ); and you can use your existing IAM user to generate these Signed URLs/Signed Cookies.