加载 AWS CloudFront 文件时出现 403（禁止）

Question

I'm working on a video app and storing the files on AWS S3, using the default URL like https://***.amazonaws.com/*** works fine but I have decided to use CloudFront which is faster for content delivery.

Using CF, I keep getting 403 (Forbidden) using this URL https://***.cloudfront.net/*** . Did I miss anything?

Everything works fine until I decide to load the contents from CloudFront which points to my bucket.

Any solution please?

Answer 1

When restricting access to S3 content using a bucket policy that inspects the incoming Referer: header, you need to do a little bit of custom configuration to "outsmart" CloudFront.

It's important to understand that CloudFront is designed to be a well-behaved cache. By "well-behaved," I mean that CloudFront is designed to never return a response that differs from what the origin server would have returned. I'm sure you can see that is an important factor.

Let's say I have a web server (not S3) behind CloudFront, and my web site is designed so that it returns different content based on an inspection of the Referer: header... or any other http request header, like User-Agent: for example. Depending on your browser, I might return different content. How would CloudFront know this, so that it would avoid serving a user the wrong version of a certain page?

The answer is, it wouldn't be able to tell -- it can't know this. So, CloudFront's solution is not to forward most request headers to my server at all. What my web server can't see, it can't react to, so the content I return cannot vary based on headers I don't receive, which prevents CloudFront from caching and returning the wrong response, based on those headers. Web caches have an obligation to avoid returning the wrong cached content for a given page.

"But wait," you object. "My site depends on the value from a certain header in order to determine how to respond." Right, that makes sense... so we have to tell CloudFront this:

Instead of caching my pages based on just the requested path, I need you to also forward the Referer: or User-Agent: or one of several other headers as sent by the browser, and cache the response for use on other requests that include not only the same path, but also the same values for the extra header(s) that you forward to me .

However, when the origin server is S3, CloudFront doesn't support forwarding most request headers, on the assumption that since static content is unlikely to vary, these headers would just cause it to cache multiple identical responses unnecessarily.

Your solution is not to tell CloudFront that you're using S3 as the origin. Instead, configure your distribution to use a "custom" origin, and give it the hostname of the bucket to use as the origin server hostname.

Then, you can configure CloudFront to forward the Referer: header to the origin, and your S3 bucket policy that denies/allows requests based on that header will work as expected.

Well, almost as expected. This will lower your cache hit ratio somewhat, since now the cached pages will be cached based on path + referring page. It an S3 object is referenced by more than one of your site's pages, CloudFront will cache a copy for each unique request. It sounds like a limitation, but really, it's only an artifact of proper cache behavior -- whatever gets forwarded to the back-end, almost all of it, must be used to determine whether that particular response is usable for servicing future requests.

See http://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/distribution-web-values-specify.html#DownloadDistValuesForwardHeaders for configuring CloudFront to whitelist specific headers to send to your origin server.

Important: don't forward any headers you don't need, since every variant request reduces your hit rate further. Particularly when using S3 as the back-end for a custom origin, do not forward the Host: header, because that is probably not going to do what you expect. Select the Referer: header here, and test. S3 should begin to see the header and react accordingly.

Note that when you removed your bucket policy for testing, CloudFront would have continued to serve the cached error page unless you flushed your cache by sending an invalidation request, which causes CloudFront to purge all cached pages matching the path pattern you specify, over the course of about 15 minutes. The easiest thing to do when experimenting is to just create a new CloudFront distribution with the new configuration, since there is no charge for the distributions themselves.

When viewing the response headers from CloudFront, note the X-Cache: (hit/miss) and Age: (how long ago this particular page was cached) responses. These are also useful in troubleshooting.

---

Update: @alexjs has made an important observation: instead of doing this using the bucket policy and forwarding the Referer: header to S3 for analysis -- which will hurt your cache ratio to an extent that varies with the spread of resources over referring pages -- you can use the new AWS Web Application Firewall service, which allows you to impose filtering rules against incoming requests to CloudFront, to allow or block requests based on string matching in request headers .

For this, you'd need to connect the distribution to S3 as as S3 origin (the normal configuration, contrary to what I proposed, in the solution above, with a "custom" origin) and use the built-in capability of CloudFront to authenticate back-end requests to S3 (so the bucket contents aren't directly accessible if requested from S3 directly by a malicious actor).

See https://www.alexjs.eu/preventing-hotlinking-using-cloudfront-waf-and-referer-checking/ for more on this option.

Answer 2

Also, it may be something simple. When you first upload a file to an S3 bucket, it is non-public, even if other files in that bucket are public, and even if the bucket itself is public.

To change this in the AWS Console, check the box next to the folder that you want to make public (the folder you just uploaded), and choose "Make public" from the menu.

The files in that folder (and any subfolders) will be made public, and you'll be able to serve the files from S3.

For the AWS CLI, add the "--acl public-read" option in your command, like so:

aws s3 cp index.html s3://your.remote.bucket --acl public-read

Answer 3

I identified another reason why CloudFront can return a 403 (Bad request) . Maybe thats an edge case but I would like to share with you.

CloudFront implements a forward loop detection mechanism to prevent from forwarding-loop attacks.
You cannot cascade more than 2 CloudFront distributions as orgins according to the AWS support.

Lets assume you have configured CloudFront A with CloudFront B as an origin and from CloudFront B you have configured CloudFront C as an origin, and from CloudFront C you have an S3 bucket as an origin.

A --> B --> C --> S3 bucket (can return a 403 error)

If you request a file from CloudFront A that is located in the S3 bucket at the end of the cascade, the CloudFront C will return a 403 (Bad request).

If your cascade just consists of 2 CloudFront distributions and an S3 bucket at the end, the request of a file from the S3 origin works.

A --> B --> S3 bucket (works)

Answer 4

For me, I had to give CodePipeline access to my S3 bucket policy. For example something like this:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "PublicReadGetObject",
            "Effect": "Allow",
            "Principal": "*",
            "Action": "s3:GetObject",
            "Resource": "arn:aws:s3:::mys3bucket/*"
        }
    ]
}

Answer 5

My requirement was to make bucket private so i used OAI, the main issue i faced was i created OAI before distribution creation and choose it in origin section dropdown and cloudfront started throwing me 403. I fixed this by letting cloudfront create OAI while creating cloudfront origin (i chose the origin domain name from dropdown and selected the bucket then it gave option to restrict s3 bucket, then you will get option to create Origin Access Identity and one more option called Grant Read Permissions on Bucket, let aws/cloudfront handle it)

sometimes aws might fail to add permission for OAI in s3 bucket, use this document to add permission manually

https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/private-content-restricting-access-to-s3.html#private-content-granting-permissions-to-oai

Also makesure you have given the entry point in both s3 and cloudfront (index.html in my case)

i have not created any error pages in cloudfront, hope it saves someones time

Edit: Reloading page was throwing 403 error, so i added error pages for 403 and 404 and page as "/index.html" in cloudfront

Answer 6

I was getting 403 error from cloudfront for POST requests, where my origin was a domain name instead of an s3 bucket.

The reason was that POST is not allowed by default by cloudfront. I enabled POST from the Behaviors tab in the console, and then it worked.

Answer 7

一个问题可能是您没有指定 CNAME（特定的或通配符），当您尝试使用域名时，它不起作用，但可以使用 CF Distro url

Answer 8

I was facing similar issue, but in my case in my bucket policy, I had mentioned only bucket ARN in resources section. Instead of that I needed to mention bucketname/* to allow access to all objects in that bucket. Thought It might be helpful for some people who are facing similar issue.

Answer 9

I resolved by updating the origin domain under my cloudfront distribution

Under origin tabs edit the origin name don't select the bucket name from the list directly rather copy the static website hosting from your s3 bucket (check under properties tab)

test.uk.s3-website.eu-west-2.amazonaws.com

Answer 10

In my case I'm using subfolders in the same s3 bucket to deploy multiple react applications. This is what I've done.

1. make sure the s3 policy has "Resource": "arn:aws:s3:::s3bucketname/ ", the " " is important.
2. on the CloudFront distribution, under origins tab, make sure you don't select from the dropdown, rather just copy the website hosting endpoint from the S3 properties tab without the http://.
3. invalidate with "/*" if necessary.