Amazon S3 和 Cloudfront 缓存，如何清除缓存或同步它们的缓存

Question

I have a cron job that runs every 10 minutes and updates the content-type and x-amz-meta. But since yesterday it seems like after the cron job run, Amazon is not picking up the changes made (refreshing his cache).

I even went and made the changes manually but no change...

When a video is uploaded it has a application/x-mp4 content-type and the cron job changes it to video/mp4 .

Although S3 has the right content type video/mp4 cloudfront shows application/x-mp4 (old content-type) ....

The cron job has been working for the last 6 months without a problem.

What is wrong with amazon caching? How can i synchronize the caching?

Answer 1

Use Invalidations to clear the cache, you can put the path to the files you want to clear, or simply use wild cards to clear everything.

http://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/Invalidation.html#invalidating-objects-api

This can also be done using the API! http://docs.aws.amazon.com/cloudfront/latest/APIReference/API_CreateInvalidation.html

The AWS PHP SDK now has the methods but if you want to use something lighter check out this library: http://www.subchild.com/2010/09/17/amazon-cloudfront-php-invalidator/

user3305600's solution doesn't work as setting it to zero is the equivalent of Using the Origin Cache Headers.

Answer 2

As to the actual code

get your CloudFront distribution id

aws cloudfront list-distributions

Invalidate all files in the distribution, so CloudFront fetches fresh ones

aws cloudfront create-invalidation --distribution-id=S11A16G5KZMEQD --paths /

My actual full release script is

#!/usr/bin/env bash

BUCKET=mysite.com
SOURCE_DIR=dist/

export AWS_ACCESS_KEY_ID=xxxxxxxxxxx
export AWS_SECRET_ACCESS_KEY=xxxxxxxxx
export AWS_DEFAULT_REGION=eu-west-1


echo "Building production"
if npm run build:prod ; then
   echo "Build Successful"
else
  echo "exiting.."
  exit 1
fi


echo "Removing all files on bucket"
aws s3 rm s3://${BUCKET} --recursive


echo "Attempting to upload site .."
echo "Command:  aws s3  sync $SOURCE_DIR s3://$BUCKET/"
aws s3  sync ${SOURCE_DIR} s3://${BUCKET}/
echo "S3 Upload complete"

echo "Invalidating cloudfrond distribution to get fresh cache"
aws cloudfront create-invalidation --distribution-id=S11A16G5KZMEQD --paths / --profile=myawsprofile

echo "Deployment complete"  

References

http://docs.aws.amazon.com/cli/latest/reference/cloudfront/get-invalidation.html

http://docs.aws.amazon.com/cli/latest/reference/cloudfront/create-invalidation.html

Answer 3

Here is a manual way to invalidate the cache for all files on CloudFront via AWS

Answer 4

S3 is not used for real time development but if you really want to test your freshly deployed website use

http://yourdomain.com/index.html?v=2
http://yourdomain.com/init.js?v=2

Adding a version parameter in the end will stop using the cached version of the file and the browser will get a fresh copy of the file from the server bucket

Answer 5

Cloudfront will cache a file/object until the cache expiry. By default it is 24 hrs. If you have changed this to a large value, then it takes longer.

If you anytime needs to force clear the cache, use the invalidation . It is charged separately.

Another option is to change the URL (object key), so it fetches the new object always.

Answer 6

If you're looking for a minimal solution that invalidates the cache, this edited version of Dr Manhattan's solution should be sufficient. Note that I'm specifying the root / directory to indicate I want the whole site refreshed.

export AWS_ACCESS_KEY_ID=<Key>
export AWS_SECRET_ACCESS_KEY=<Secret>
export AWS_DEFAULT_REGION=eu-west-1

echo "Invalidating cloudfrond distribution to get fresh cache"
aws cloudfront create-invalidation --distribution-id=<distributionId> --paths / --profile=<awsprofile>

Region Codes can be found here

You'll also need to create a profile using the aws cli. Use the aws configure --profile option. Below is an example snippet from Amazon.

$ aws configure --profile user2
AWS Access Key ID [None]: AKIAI44QH8DHBEXAMPLE
AWS Secret Access Key [None]: je7MtGbClwBF/2Zp9Utk/h3yCo8nvbEXAMPLEKEY
Default region name [None]: us-east-1
Default output format [None]: text

Answer 7

(edit: Does not work)As of 2014, You can clear your cache whenever you want, Please go thorough the Documentation or just go to your distribution settings>Behaviors>Edit

Object Caching Use ( Origin Cache Headers ) Customize

Minimum TTL = 0

http://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/Expiration.html

Answer 8

Don't use invalidations. They cannot be reverted and you will be charged. They only way it works for me is reducing the TTL and wait.

Regards

Answer 9

I believe using * invalidate the entire cache in the distribution. I am trying at the moment, I would update it further

invalidate request screenshot

Update:

It worked as expected. Please note that you can invalidate the object you would like by specifying the object path.