How to set nginx proxy path to use static contents of a specific s3 bucket folder?

Question

I'm working on an nginx reverse proxy container image to proxy frontend files from s3, and Im trying to access these files from a specific folder location, instead of just the base path of the s3 bucket. As of yet I can only serve up the index.html which I'm using a rewrite for, but I'm getting a 403 on the js and css files.

I've tried including mime.types

include       mime.types;

I've tried adding an s3 folder bucket param

proxy_pass http://YOURBUCKET.s3-website.eu-central-1.amazonaws.com/$1;

And then various regex options

Here is my nginx conf file

server {
    listen 80;
    listen  443 ssl;
    ssl_certificate     /etc/ssl/nginx-server.crt;
    ssl_certificate_key   /etc/ssl/nginx-server.key;

    server_name timemachine.com;
    sendfile on;
    default_type application/octet-stream;
    resolver        8.8.8.8;
    server_tokens   off;

    location ~ ^/app1/(.*) {
        set $s3_bucket_endpoint  "timemachineapp.s3-us-east-1.amazonaws.com";
        proxy_http_version     1.1;
        proxy_buffering        off;
        proxy_ignore_headers   "Set-Cookie";
        proxy_hide_header      x-amz-id-2;
        proxy_hide_header      x-amz-request-id;
        proxy_hide_header      x-amz-meta-s3cmd-attrs;
        proxy_hide_header      Set-Cookie;
        proxy_set_header       Authorization "";
        proxy_intercept_errors on;
        rewrite ^/app1/?$ /dev/app1/index.html;    <-- I can only access index.html and the other js and css files throw a 403
        proxy_pass https://timemachineapp.s3-us-east-1.amazonaws.com;
        break;
    }
}

As you can see, I'm trying to make this so that the user goes to https://timemachine/app1 that this will go to the homepage and load all the css and js files. Again, what im getting is a 403 and sometimes a 404. Insight appreciated.

Answer 1

From the question it looks like

• There's a constant request-url prefix /app1/
• There's a constant proxied-url prefix /dev/app1/

On that basis...

First, enable the debug log

There will already be an error_log directive in the nginx config, locate it and temporarily change to debug:

error_log  /dev/stderr debug;

This will allow you to see how these requests are being processed.

Try naive-simple first

Let's use this config (other header directives omitted for brevity):

location = /app1 { # redirect for consistency
    return 301 /app1/;
}

location = /app1/ { # explicitly handle the 'index' request
    proxy_pass https://example.com/dev/app1/index.html;
}

location /app1/ {
    proxy_pass https://example.com/dev/;
}

And emit a request to it:

$ ~ curl -I http://test-nginx/app1/some/path/some-file.txt
HTTP/1.1 403 Forbidden
...

Note that S3 returns a 403 for requests that don't exist, nginx is just proxying that response here.

Let's look in the logs to see what happened:

2023/01/28 14:46:10 [debug] 15#0: *1 test location: "/"
2023/01/28 14:46:10 [debug] 15#0: *1 test location: "app1/"
2023/01/28 14:46:10 [debug] 15#0: *1 using configuration "/app1/"
...
"HEAD /dev/some/path/some-file.txt HTTP/1.0
Host: example.com
Connection: close
User-Agent: curl/7.79.1
Accept: */*
"

So our request became https://example.com/dev/some/path/some-file.txt

That's because the way proxy_pass works is:

If the proxy_pass directive is specified with a URI, then when a request is passed to the server, the part of a normalized request URI matching the location is replaced by a URI specified in the directive

Meaning:

Nginx receives:
/app1/some/path/some-file.txt
      ^ the normalized path starts here

Proxied-upstream receives:
/dev/some/path/some-file.txt
     ^ and was appended to proxy-pass URI

I point this out as renaming/moving things on s3 may lead to a simpler nginx setup.

Rewrite all paths, not specific requests

Modifying the config above like so:

location = /app1 { # redirect for consistency
    return 301 /app1/;
}

location = /app1/ { # explicitly handle the 'index' request
    proxy_pass https://example.com/dev/app1/index.html;
}

location /app1/ {
    rewrite ^/(.*) /dev/$1 break;             # prepend with /dev/
    # rewrite ^/app1/(.*) /dev/app1/$1 break; # OR this
    proxy_pass https://example.com/;          # no path here
}

And trying that test-request again yields the following logs:

"HEAD /dev/app1/some/path/some-file.txt HTTP/1.0
Host: example.com
Connection: close
User-Agent: curl/7.79.1
Accept: */*

"

In this way the index request works, but also arbitrary paths - and there's no need to modify this config to handle each individual url requested.

Answer 2

Alright so found a solution. Unless I'm missing something, this is easier than thought. For my use case, all I had to do was simply add multiple writes with those css files passed in (I'm sure there's a simpler way to just specify any.css file extension regardless of the naming of the file. Anyway, here is solution at the moment:

server {
    listen 80;
    listen  443 ssl;
    ssl_certificate     /etc/ssl/nginx-server.crt;
    ssl_certificate_key   /etc/ssl/nginx-server.key;

    server_name timemachine.com;
    sendfile on;
    default_type application/octet-stream;
    resolver        8.8.8.8;
    server_tokens   off;

    location ~ ^/app1/(.*) {
        set $s3_bucket_endpoint  "timemachineapp.s3-us-east-1.amazonaws.com";
        proxy_http_version     1.1;
        proxy_buffering        off;
        proxy_ignore_headers   "Set-Cookie";
        proxy_hide_header      x-amz-id-2;
        proxy_hide_header      x-amz-request-id;
        proxy_hide_header      x-amz-meta-s3cmd-attrs;
        proxy_hide_header      Set-Cookie;
        proxy_set_header       Authorization "";
        proxy_intercept_errors on;
        rewrite ^/app1/?$ /dev/app1/index.html;   
        rewrite ^/app1/?$ /dev/app1/cssfile.css;  <- and keep adding, if needed
        proxy_pass https://timemachineapp.s3-us-east-1.amazonaws.com;
        break;
    }
}