了解服务器架构：使用 Nginx 反向代理或 Apache 服务器从 AWS S3 传送内容

Question

The purpose of this question is to understand the strategy while designing server side architecture.

Use case: I want to build a http server for an app which allow users to upload and download multimedia content (images, videos etc.) Large number of concurrent users (say, around 50k) are expected to upload/download the content.

All the content will be stored on AWS S3 bucket. Information regarding S3 bucket ie bucket name/authentication headers should be masked from the user. Since there are multiple Access Control Options ( AWS-ACL ) for S3 bucket, it would be preferable to refrain from making the bucket available for All_Users (authenticated and anonymous users). I do not want to expose the content in public domain.

Queries

• Since I want to mask AWS S3 from the users, I will need to use a web-server or reverse proxy. I have gone through multiple resources that compare Apache Vs Nginx. Since the server needs to deliver static content from S3 to high number of concurrent users, Nginx seems to be a better option. Isn't it??

• Does setting Access Control Level to S3 bucket to ALL_USERS ( to authenticated and anonymous users) compromise on data privacy? If I use reverse proxy, there is no way for the user to determine S3 bucket urls. Is the data safe and private?

• However, if S3 bucket is made available for Authenticated users only, will nginx reverse proxy work? I have gone through Nginx Reverse Proxy for S3 . In order for Nginx to work as a reverse proxy, a Pre-signed URL needs to be prepared. The expiry time of pre-signed url is again a tricky decision. Does setting a huge expiry time for pre-signed url makes sense? Does it compromise on the security or privacy of data (similar to s3 access control to ALL_USERS)? If yes, is there a way to reverse proxy the request to dynamically generated pre-signed url (with short expiry time) via nginx only?

Any information and resources to consolidate my understanding will be really helpful.

Answer 1

Does setting Access Control Level to S3 bucket to ALL_USERS ( to authenticated and anonymous users) compromise on data privacy?

Absolutely. Don't do it.

If I use reverse proxy, there is no way for the user to determine S3 bucket urls. Is the data safe and private?

Theoretically, they can't determine it, but what if an error message or misconfiguration leaks the information? This is security through obscurity , which gives you nothing more than a false sense of security. There's always a better way.

Information regarding S3 bucket ie bucket name/authentication headers should be masked from the user.

The authentication mechanism of S3, with signed URLs, is designed so that there is no harm in exposing it to the user. The only thing secret is your AWS Secret Key, which you'll note is not exposed in a signed URL. It also can't reasonably be reverse-engineered, and a signed URL is good for only the resource and action that the signature permits.

Signing URLs and presenting them to the user does not pose a security risk, although, admittedly, there are other reasons why you might not want to do that. I do that routinely -- signing a URL while a page is being rendered, with a relatively long expiration time, or signing a URL and redirecting a user to the signed URL when they click on a link back to my application server (which validates their authorization to access the resource, and then returns a signed URL with a very short expiration time, such as 5 to 10 seconds; the expiration can occur while a download is in progress without causing a problem -- the signature only needs to avoid expiring before the request to S3 is accepted).

However, if you want to go the proxy route (which, in addition to the above, is something I do in my systems as well), there's a much easier way than what you're envisioning: the bucket policy can be configured to permit specific permissions to be granted based on source IP addresses... of your servers.

Here's a (sanitized) policy taken directly from one of my buckets. The IP addresses are from RFC-5737 to avoid the confusion that private IP addresses in this example would cause.

These IP addresses are public IP addresses... they would be your elastic IP addresses attached to your web servers, or, preferably, to the NAT instances that the web servers use for their outgoing requests.

{
    "Version": "2008-10-17",
    "Id": "Policy123456789101112",
    "Statement": [
        {
            "Sid": "Stmt123456789101112",
            "Effect": "Allow",
            "Principal": {
                "AWS": "*"
            },
            "Action": "s3:GetObject",
            "Resource": "arn:aws:s3:::example-bucket/*",
            "Condition": {
                "IpAddress": {
                    "aws:SourceIp": [
                        "203.0.113.173/32",
                        "203.0.113.102/32",
                        "203.0.113.52/32",
                        "203.0.113.19/32"
                    ]
                }
            }
        }
    ]
}

What does this do? If a request arrives at S3 from one of the listed IP addresses, the GetObject permission is granted to the requester. With a proxy, your proxy's IP address will be the IP address seen by S3, and the request will be granted if it matches the bucket policy, allowing your proxies to fetch objects from S3 while not allowing the rest of the Internet to, unless alternate credentials are presented, such as with a signed URL. This policy doesn't "deny" anything directly, because the deny is implicit. Importantly, don't upload your objects with the public-read ACL, because that would allow the objected to be downloaded by anyone. The default private ACL works perfectly for this application.

S3 can grant permissions like this based on other criteria, such as the Referer: header, and you may find examples of that online, but don't do that . Trusting what the browser reports as the referring page is an extremely weak and primitive security mechanism that provides virtually no real protection -- headers are incredibly simple to spoof. That sort of filtering is really only good for annoying lazy people who are hot-linking to your content. The source IP address is a different matter altogether, as it's not carried in a layer 7 header, and cannot be readily spoofed.

Because S3 only interacts with the Internet via the TCP protocol, your source addresses -- even it it were known how you had enabled the bucket to trust these addresses -- cannot be spoofed in any practical way, because to do so would mean to breach the security of AWS's core IP network infrastructure -- TCP requires the originating machine to be reachable across subnet boundaries by the source IP address it uses, and the AWS network would only ever route those responses back to your legitimately-allocated IP address, which would have no option other than to reset or discard the connections, since they were not initiated with you.

Note that this solution does not work in conjunction with S3 VPC endpoints which Amazon recently announced , because with S3 VPC endpoints, your source IP address (seen by S3) will be the private address, which isn't unique to your VPC... but that should not be a problem. I mention this caveat only in the interest of thoroughness. S3 VPC endpoints are not required and not enabled by default, and if enabled, can be provisioned on a per-subnet basis.