nginx添加新站点会导致ERR_TOO_MANY_REDIRECTS

Question

I'm trying to add another Django app to my server. I already have the xmlanalyzer.maciejg.pl up&running, now I'm trying to add to the existing nginx & gunicorn setup another app to be available at fencing.maciejg.pl.

I've used the existing XMLAnalyzer gunicorn setup (working fine):

#!/bin/bash

NAME="xmlanalyzer"                                   # Name of the application
DJANGODIR=/home/django/xmlanalyzer                   # Django project directory
SOCKFILE=/home/django/xmlanalyzer/run/gunicorn.sock  # we will communicte using this unix socket
USER=my-user-name                                             # the user to run as
GROUP=my-user-name                                            # the group to run as
NUM_WORKERS=3                                        # how many worker processes should Gunicorn spawn
DJANGO_SETTINGS_MODULE=xmlanalyzer.settings          # which settings file should Django use
DJANGO_WSGI_MODULE=xmlanalyzer.wsgi                  # WSGI module name

echo "Starting $NAME as `whoami`"

# Activate the virtual environment
cd $DJANGODIR
source ../venv/bin/activate
export DJANGO_SETTINGS_MODULE=$DJANGO_SETTINGS_MODULE
export PYTHONPATH=$DJANGODIR:$PYTHONPATH

# Create the run directory if it doesn't exist
RUNDIR=$(dirname $SOCKFILE)
test -d $RUNDIR || mkdir -p $RUNDIR

# Start your Django Unicorn
# Programs meant to be run under supervisor should not daemonize themselves (do not use --daemon)
#exec gunicorn -b 127.0.0.1:8001 xmlanalyzer.wsgi:application --pid /tmp/gunicorn.pid ;
exec gunicorn -b 127.0.0.1:8001 ${DJANGO_WSGI_MODULE}:application \
  --name $NAME \
  --workers $NUM_WORKERS \
  --user=$USER --group=$GROUP \
##   --bind=unix:$SOCKFILE \
  --bind=127.0.0.1:8001 \
  --log-level=debug \
  --log-file=-

Here's gunicorn setup for Fencing app (not working):

#!/bin/bash

NAME="fencing"                                  # Name of the application
DJANGODIR=/home/django/fencing                  # Django project directory
SOCKFILE=/home/django/fencing/run/gunicorn.sock # we will communicte using this unix socket
USER=my-user-name                                        # the user to run as
GROUP=my-user-name                                       # the group to run as
NUM_WORKERS=3                                   # how many worker processes should Gunicorn spawn
DJANGO_SETTINGS_MODULE=mysite.settings          # which settings file should Django use
DJANGO_WSGI_MODULE=mysite.wsgi                  # WSGI module name

echo "Starting $NAME as `whoami`"

# Activate the virtual environment
cd $DJANGODIR
source ../venv/bin/activate
export DJANGO_SETTINGS_MODULE=$DJANGO_SETTINGS_MODULE
export PYTHONPATH=$DJANGODIR:$PYTHONPATH

# Create the run directory if it doesn't exist
RUNDIR=$(dirname $SOCKFILE)
test -d $RUNDIR || mkdir -p $RUNDIR

# Start your Django Unicorn
# Programs meant to be run under supervisor should not daemonize themselves (do not use --daemon)

exec gunicorn -b 127.0.0.1:8002 ${DJANGO_WSGI_MODULE}:application \
  --name $NAME \
  --workers $NUM_WORKERS \
  --user=$USER --group=$GROUP \
##   --bind=unix:$SOCKFILE \
  --bind=127.0.0.1:8002 \
  --log-level=debug \
  --log-file=-

Both seem to be up:

ps -ef | grep gunicorn
my-user-name        780 20697  0 10:20 ?        00:00:01 /home/django/venv/bin/python3 /home/django/venv/bin/gunicorn -b 127.0.0.1:8002 mysite.wsgi:application --name fencing --workers 3 --user=my-user-name --group=my-user-name
my-user-name        787   780  0 10:20 ?        00:00:00 /home/django/venv/bin/python3 /home/django/venv/bin/gunicorn -b 127.0.0.1:8002 mysite.wsgi:application --name fencing --workers 3 --user=my-user-name --group=my-user-name
my-user-name        788   780  0 10:20 ?        00:00:00 /home/django/venv/bin/python3 /home/django/venv/bin/gunicorn -b 127.0.0.1:8002 mysite.wsgi:application --name fencing --workers 3 --user=my-user-name --group=my-user-name
my-user-name        789   780  0 10:20 ?        00:00:00 /home/django/venv/bin/python3 /home/django/venv/bin/gunicorn -b 127.0.0.1:8002 mysite.wsgi:application --name fencing --workers 3 --user=my-user-name --group=my-user-name
my-user-name       1712  1656  0 12:40 pts/1    00:00:00 grep --color=auto gunicorn
root      1730     1  0  2018 ?        01:04:09 /home/django/venv/bin/python3 /home/django/venv/bin/gunicorn -b 127.0.0.1:8001 xmlanalyzer.wsgi:application --name xmlanalyzer --workers 3 --user=my-user-name --group=my-user-name
my-user-name      17483  1730  0 Mar25 ?        00:01:12 /home/django/venv/bin/python3 /home/django/venv/bin/gunicorn -b 127.0.0.1:8001 xmlanalyzer.wsgi:application --name xmlanalyzer --workers 3 --user=my-user-name --group=my-user-name
my-user-name      17554  1730  0 Mar25 ?        00:01:05 /home/django/venv/bin/python3 /home/django/venv/bin/gunicorn -b 127.0.0.1:8001 xmlanalyzer.wsgi:application --name xmlanalyzer --workers 3 --user=my-user-name --group=my-user-name
my-user-name      17953  1730  0 Mar25 ?        00:00:41 /home/django/venv/bin/python3 /home/django/venv/bin/gunicorn -b 127.0.0.1:8001 xmlanalyzer.wsgi:application --name xmlanalyzer --workers 3 --user=my-user-name --group=my-user-name

I've created nginx setup using the following:

/etc/nginx/sites-available# more xmlanalyzer
server {
    server_name xmlanalyzer.maciejg.pl;

    access_log off;

    location /static/ {
        alias /home/django/xmlanalyzer/XMLAnalyzer/static/;
    }

    location / {
        proxy_set_header Host $host;
        proxy_pass http://127.0.0.1:8001;
        proxy_set_header X-Forwarded-Host $server_name;
        proxy_set_header X-Real-IP $remote_addr;
        add_header P3P 'CP="ALL DSP COR PSAa PSDa OUR NOR ONL UNI COM NAV"';
    }
 # managed by Certbot

    listen 443 ssl; # managed by Certbot
    ssl_certificate /etc/letsencrypt/live/xmlanalyzer.maciejg.pl/fullchain.pem; # managed by Certbot
    ssl_certificate_key /etc/letsencrypt/live/xmlanalyzer.maciejg.pl/privkey.pem; # managed by Certbot
    include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot

}

server {
    if ($host = xmlanalyzer.maciejg.pl) {
        return 301 https://$host$request_uri;
    } # managed by Certbot


    server_name xmlanalyzer.maciejg.pl;
    listen 80;
    return 404; # managed by Certbot
    client_max_body_size 64M;
}

I've changed the folders and the port from 8001 to 8002. As a result I've got:

/etc/nginx/sites-available# more fencing
server {
    server_name fencing.maciejg.pl;

    access_log off;

    location = /favicon.ico { access_log off; log_not_found off; }
    location /static/ {
        alias /home/django/fencing/fencingtournament/static/;
    }

    access_log /home/django/fencing/logs/nginx-access.log;
    error_log /home/django/fencing/logs/nginx-error.log;

    location / {
        proxy_set_header Host $host;
        proxy_pass http://127.0.0.1:8002;
        proxy_set_header X-Forwarded-Host $server_name;
        proxy_set_header X-Real-IP $remote_addr;
        add_header P3P 'CP="ALL DSP COR PSAa PSDa OUR NOR ONL UNI COM NAV"';
    }
 # managed by Certbot

    listen 443 ssl; # managed by Certbot
    ssl_certificate /etc/letsencrypt/live/fencing.maciejg.pl/fullchain.pem; # managed by Certbot
    ssl_certificate_key /etc/letsencrypt/live/fencing.maciejg.pl/privkey.pem; # managed by Certbot
    include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot

}
server {
    if ($host = fencing.maciejg.pl) {
        return 301 https://$host$request_uri;
    } # managed by Certbot


    server_name fencing.maciejg.pl;
    listen [::]:80;
    return 404; # managed by Certbot
}

Now, while https://xmlanalyzer.maciejg.pl works great, I get ERR_TOO_MANY_REDIRECTS while trying to access https://fencing.maciejg.pl/

Curl proves that domain is set up correctly and nginx is available. For some reason it does not redirect to my app:

curl fencing.maciejg.pl
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
    body {
        width: 35em;
        margin: 0 auto;
        font-family: Tahoma, Verdana, Arial, sans-serif;
    }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>
</body>
</html>

Both sites are available.

ll ../sites-enabled/
total 12
drwxr-xr-x 2 root root 4096 Mar 25 20:39 ./
drwxr-xr-x 6 root root 4096 Mar 25 22:59 ../
lrwxrwxrwx 1 root root   26 Mar 25 20:39 fencing -> ../sites-available/fencing
lrwxrwxrwx 1 root root   30 Feb 23  2018 xmlanalyzer -> ../sites-available/xmlanalyzer

gunicorn log looks good to me:

tail gunicorn-error.log
Starting fencing as my-user-name
[2019-03-26 10:20:01 +0000] [780] [INFO] Starting gunicorn 19.7.1
[2019-03-26 10:20:01 +0000] [780] [INFO] Listening at: http://127.0.0.1:8002 (780)
[2019-03-26 10:20:01 +0000] [780] [INFO] Using worker: sync
[2019-03-26 10:20:01 +0000] [787] [INFO] Booting worker with pid: 787
[2019-03-26 10:20:01 +0000] [788] [INFO] Booting worker with pid: 788
[2019-03-26 10:20:02 +0000] [789] [INFO] Booting worker with pid: 789

nginx-access.log and nginx-error.log in my /home/django/fencing/logs foler are empty.

What did I miss? I'll appreciate any input.

EDIT Just to add - the app can be accessed if started manually:

Starting development server at http://159.65.24.62:8002/
[26/Mar/2019 14:24:37] You're accessing the development server over HTTPS, but it only supports HTTP.

The error here is expected - this is just to show that a web request to https://fencing.maciejg.pl:8002/ did hit the right spot, so the domain is set up ok.

EDIT 2 Changed nginx setup for Fencing app to access_log on; with the result as follows:

more nginx-access.log

37.30.26.37 - - [26/Mar/2019:14:14:04 +0000] "GET / HTTP/1.1" 301 5 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko
) Chrome/73.0.3683.86 Safari/537.36"
37.30.26.37 - - [26/Mar/2019:14:14:04 +0000] "GET / HTTP/1.1" 301 5 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko
) Chrome/73.0.3683.86 Safari/537.36"

So I can see that the request reaches nginx and it is refusing.

EDIT 3 After disabling redirects with certbot I now get nginx landing page when visiting http://fencing.maciejg.pl/ - so again, it is coming through, just not redirected to my app run by gunicorn . So I still believe this is a bug in my nginx setup - yet, I still do not see it...

Answer 1

Ok, I finally got this one resolved! There were multiple issues.

First, I had a collision in nginx setup. The though part was, even once I got this one resolved, I still got HTTP errors. This was due to the fact, that gunicorn setup was invalid. The though part was, even once I got this one resolved, I still got HTTP errors. This was due to the fact, that Django setup was invalid.

So I messed around with every piece round and round again and my advice and recipee for resolving would be: do a step by step analysis.

First: I've run my Django app in development mode by starting python manage.py runserver 127.0.0.1:8000 with Debug = True and SSL disabled. This way with curl I was able to get it responding and confirm it works.

Next I've killed the app and run it via gunicorn script. I found out that it was throwing an error due to the fact that the environmental variable holding the SECRET_KEY was not available. Honestly - I didn't get this one resolved, I've switched to keeping the key in a separate file. So, one issue resolved, I got my gunicorn running fine.

Nest step: kill the gunicorn process and invoke it using supervisiord . Turned out there was a privilige issue as supervisiord runs on a different account.

Once I got that one resolved, I've recreated my nginx setup from a scratch till it started forwarding the requests. No SSL.

Next, I rerun the certbot to put SSL back. Here I also found out that you need to clear browser cache as it did not show the page even though the setup underneath was fine. This took a while as well - I've accidentaly discovered this one as I tried reaching my site on a differnt laptop to discover, that it works fine.

Finally, I put my other nginx server configurations in sites-enabled back, one by one.

I hope I will never need this solution, and that it will only be useful to others :)