502 Bad Gateway nginx/1.20.0 Error when Deploying Django app using AWS Elastic Beanstalk

Question

I was following closely following AWS's documentation on how to deploy Django apps using Elastic Beanstalk until I received a 502 Bad Gateway nginx/1.20.0 Error when opening my app. From there, I followed a couple of solutions offered by users that suffered similar problems. Specifically, I added a Procfile and went about editing how my django.config file was set up. However, the error still remains and I would like some guidance on how to tackle this problem. Below are the relevant logs and files:

eb-engine.log

django.config

option_settings:
  aws:elasticbeanstalk:container:python:
    WSGIPath: djangoproject.wsgi:application

Procfile

web: gunicorn --bind :8000 --workers 3 --threads 2 djangoproject.wsgi:application

settings.py

"""
Django settings for djangoproject project.

Generated by 'django-admin startproject' using Django 3.1.7.

For more information on this file, see
https://docs.djangoproject.com/en/3.1/topics/settings/

For the full list of settings and their values, see
https://docs.djangoproject.com/en/3.1/ref/settings/
"""

import os
from pathlib import Path


# Build paths inside the project like this: BASE_DIR / 'subdir'.
BASE_DIR = Path(__file__).resolve().parent.parent.parent


# Quick-start development settings - unsuitable for production
# See https://docs.djangoproject.com/en/3.1/howto/deployment/checklist/

# SECURITY WARNING: keep the secret key used in production secret!
SECRET_KEY = 'thesecuritykey'

# SECURITY WARNING: don't run with debug turned on in production!
DEBUG = True

ALLOWED_HOSTS = ['localhost','127.0.0.1','djangoproject.eba-fuipjhfa.us-west-2.elasticbeanstalk.com']


# Application definition

INSTALLED_APPS = [
    'django.contrib.admin',
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.messages',
    'django.contrib.staticfiles',
    'store.apps.StoreConfig',
    'basket.apps.BasketConfig',
    'account.apps.AccountConfig',
    'orders.apps.OrdersConfig',
    'checkout.apps.CheckoutConfig',
    'mptt',
]

MIDDLEWARE = [
    'django.middleware.security.SecurityMiddleware',
    'django.contrib.sessions.middleware.SessionMiddleware',
    'django.middleware.common.CommonMiddleware',
    'django.middleware.csrf.CsrfViewMiddleware',
    'django.contrib.auth.middleware.AuthenticationMiddleware',
    'django.contrib.messages.middleware.MessageMiddleware',
    'django.middleware.clickjacking.XFrameOptionsMiddleware',
]

ROOT_URLCONF = 'djangoproject.urls'

TEMPLATES = [
    {
        'BACKEND': 'django.template.backends.django.DjangoTemplates',
        'DIRS': [BASE_DIR / 'templates'],
        'APP_DIRS': True,
        'OPTIONS': {
            'context_processors': [
                'django.template.context_processors.debug',
                'django.template.context_processors.request',
                'django.contrib.auth.context_processors.auth',
                'django.contrib.messages.context_processors.messages',
                'store.context_processors.categories',
                'basket.context_processor.basket'
            ],
        },
    },
]

WSGI_APPLICATION = 'djangoproject.wsgi.application'


# Database
# https://docs.djangoproject.com/en/3.1/ref/settings/#databases

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.sqlite3',
        'NAME': BASE_DIR / 'db.sqlite3',
    }
}


# Password validation
# https://docs.djangoproject.com/en/3.1/ref/settings/#auth-password-validators

AUTH_PASSWORD_VALIDATORS = [
    {
        'NAME': 'django.contrib.auth.password_validation.UserAttributeSimilarityValidator',
    },
    {
        'NAME': 'django.contrib.auth.password_validation.MinimumLengthValidator',
    },
    {
        'NAME': 'django.contrib.auth.password_validation.CommonPasswordValidator',
    },
    {
        'NAME': 'django.contrib.auth.password_validation.NumericPasswordValidator',
    },
]


# Internationalization
# https://docs.djangoproject.com/en/3.1/topics/i18n/

LANGUAGE_CODE = 'en-us'

TIME_ZONE = 'UTC'

USE_I18N = True

USE_L10N = True

USE_TZ = True


# Static files (CSS, JavaScript, Images)
# https://docs.djangoproject.com/en/3.1/howto/static-files/

STATIC_URL = '/static/'

STATICFILES_DIRS = [
    os.path.join(BASE_DIR, "static")
]

MEDIA_URL = '/media/'
MEDIA_ROOT = os.path.join(BASE_DIR, 'media/')

requirements.txt

nginx error logs

2021/07/26 03:24:55 [error] 6904#6904: *119 connect() failed (111: Connection refused) while connecting to upstream, client: 172.31.15.153, server: , request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:8000/", host: "52.38.207.124"
2021/07/26 03:42:04 [error] 6904#6904: *367 connect() failed (111: Connection refused) while connecting to upstream, client: 172.31.15.153, server: , request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:8000/", host: "52.38.207.124"
2021/07/26 03:51:27 [error] 6904#6904: *503 connect() failed (111: Connection refused) while connecting to upstream, client: 172.31.45.215, server: , request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:8000/", host: "44.237.64.108"
2021/07/26 03:53:58 [error] 6904#6904: *537 connect() failed (111: Connection refused) while connecting to upstream, client: 172.31.45.215, server: , request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:8000/", host: "44.237.64.108"
2021/07/26 04:00:42 [error] 6904#6904: *633 connect() failed (111: Connection refused) while connecting to upstream, client: 172.31.15.153, server: , request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:8000/", host: "52.38.207.124"
2021/07/26 04:07:37 [error] 6904#6904: *728 connect() failed (111: Connection refused) while connecting to upstream, client: 172.31.15.153, server: , request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:8000/", host: "52.38.207.124"
2021/07/26 04:08:32 [error] 6904#6904: *728 connect() failed (111: Connection refused) while connecting to upstream, client: 172.31.15.153, server: , request: "GET /.env HTTP/1.1", upstream: "http://127.0.0.1:8000/.env", host: "52.38.207.124"
2021/07/26 04:08:32 [error] 6904#6904: *728 connect() failed (111: Connection refused) while connecting to upstream, client: 172.31.15.153, server: , request: "POST / HTTP/1.1", upstream: "http://127.0.0.1:8000/", host: "52.38.207.124"
2021/07/26 04:35:42 [error] 6904#6904: *1140 connect() failed (111: Connection refused) while connecting to upstream, client: 172.31.15.153, server: , request: "GET /config/getuser?index=0 HTTP/1.1", upstream: "http://127.0.0.1:8000/config/getuser?index=0", host: "52.38.207.124"
2021/07/26 04:38:14 [error] 6904#6904: *1179 connect() failed (111: Connection refused) while connecting to upstream, client: 172.31.45.215, server: , request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:8000/", host: "44.237.64.108"
2021/07/26 04:50:45 [error] 6904#6904: *1360 connect() failed (111: Connection refused) while connecting to upstream, client: 172.31.15.153, server: , request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:8000/", host: "52.38.207.124"
2021/07/26 04:56:51 [error] 6904#6904: *1449 connect() failed (111: Connection refused) while connecting to upstream, client: 172.31.15.153, server: , request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:8000/", host: "52.38.207.124"
2021/07/26 04:56:51 [error] 6904#6904: *1449 connect() failed (111: Connection refused) while connecting to upstream, client: 172.31.15.153, server: , request: "GET /.DS_Store HTTP/1.1", upstream: "http://127.0.0.1:8000/.DS_Store", host: "52.38.207.124"
2021/07/26 04:56:52 [error] 6904#6904: *1449 connect() failed (111: Connection refused) while connecting to upstream, client: 172.31.15.153, server: , request: "GET /nginx.conf HTTP/1.1", upstream: "http://127.0.0.1:8000/nginx.conf", host: "52.38.207.124"
2021/07/26 04:56:52 [error] 6904#6904: *1449 connect() failed (111: Connection refused) while connecting to upstream, client: 172.31.15.153, server: , request: "GET /status HTTP/1.1", upstream: "http://127.0.0.1:8000/status", host: "52.38.207.124"
2021/07/26 04:56:52 [error] 6904#6904: *1449 connect() failed (111: Connection refused) while connecting to upstream, client: 172.31.15.153, server: , request: "GET /status%3E%3Cscript%3Ealert(31337)%3C%2Fscript%3E HTTP/1.1", upstream: "http://127.0.0.1:8000/status%3E%3Cscript%3Ealert(31337)%3C%2Fscript%3E", host: "52.38.207.124"
2021/07/26 04:56:53 [error] 6904#6904: *1449 connect() failed (111: Connection refused) while connecting to upstream, client: 172.31.15.153, server: , request: "GET /v2/_catalog HTTP/1.1", upstream: "http://127.0.0.1:8000/v2/_catalog", host: "52.38.207.124"
2021/07/26 04:56:53 [error] 6904#6904: *1449 connect() failed (111: Connection refused) while connecting to upstream, client: 172.31.15.153, server: , request: "GET /server-status HTTP/1.1", upstream: "http://127.0.0.1:8000/server-status", host: "52.38.207.124"
2021/07/26 04:56:53 [error] 6904#6904: *1449 connect() failed (111: Connection refused) while connecting to upstream, client: 172.31.15.153, server: , request: "GET /.env HTTP/1.1", upstream: "http://127.0.0.1:8000/.env", host: "52.38.207.124"
2021/07/26 04:56:54 [error] 6904#6904: *1449 connect() failed (111: Connection refused) while connecting to upstream, client: 172.31.15.153, server: , request: "GET /.git/config HTTP/1.1", upstream: "http://127.0.0.1:8000/.git/config", host: "52.38.207.124"
2021/07/26 04:56:54 [error] 6904#6904: *1449 connect() failed (111: Connection refused) while connecting to upstream, client: 172.31.15.153, server: , request: "GET /idx_config/ HTTP/1.1", upstream: "http://127.0.0.1:8000/idx_config/", host: "52.38.207.124"
2021/07/26 05:00:06 [error] 6904#6904: *1508 connect() failed (111: Connection refused) while connecting to upstream, client: 172.31.45.215, server: , request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:8000/", host: "44.237.64.108"
2021/07/26 05:01:21 [error] 6904#6904: *1539 connect() failed (111: Connection refused) while connecting to upstream, client: 172.31.45.215, server: , request: "GET /config/getuser?index=0 HTTP/1.1", upstream: "http://127.0.0.1:8000/config/getuser?index=0", host: "44.237.64.108"
2021/07/26 05:11:53 [error] 6904#6904: *1679 connect() failed (111: Connection refused) while connecting to upstream, client: 172.31.15.153, server: , request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:8000/", host: "52.38.207.124"
2021/07/26 05:12:31 [error] 6904#6904: *1701 connect() failed (111: Connection refused) while connecting to upstream, client: 172.31.45.215, server: , request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:8000/", host: "44.237.64.108"
2021/07/26 05:13:59 [error] 6904#6904: *1722 connect() failed (111: Connection refused) while connecting to upstream, client: 172.31.45.215, server: , request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:8000/", host: "djangoproject.eba-fuipjhfa.us-west-2.elasticbeanstalk.com"
2021/07/26 05:14:00 [error] 6904#6904: *1722 connect() failed (111: Connection refused) while connecting to upstream, client: 172.31.45.215, server: , request: "GET /favicon.ico HTTP/1.1", upstream: "http://127.0.0.1:8000/favicon.ico", host: "djangoproject.eba-fuipjhfa.us-west-2.elasticbeanstalk.com", referrer: "http://djangoproject.eba-fuipjhfa.us-west-2.elasticbeanstalk.com/"
2021/07/26 05:18:06 [error] 6904#6904: *1792 connect() failed (111: Connection refused) while connecting to upstream, client: 172.31.45.215, server: , request: "POST /vendor/phpunit/phpunit/src/Util/PHP/eval-stdin.php HTTP/1.1", upstream: "http://127.0.0.1:8000/vendor/phpunit/phpunit/src/Util/PHP/eval-stdin.php", host: "44.237.64.108"
2021/07/26 05:18:06 [error] 6904#6904: *1792 connect() failed (111: Connection refused) while connecting to upstream, client: 172.31.45.215, server: , request: "GET /vendor/phpunit/phpunit/src/Util/PHP/eval-stdin.php HTTP/1.1", upstream: "http://127.0.0.1:8000/vendor/phpunit/phpunit/src/Util/PHP/eval-stdin.php", host: "44.237.64.108"
2021/07/26 05:18:06 [error] 6904#6904: *1802 connect() failed (111: Connection refused) while connecting to upstream, client: 172.31.45.215, server: , request: "POST /api/jsonws/invoke HTTP/1.1", upstream: "http://127.0.0.1:8000/api/jsonws/invoke", host: "44.237.64.108"
2021/07/26 05:18:07 [error] 6904#6904: *1802 connect() failed (111: Connection refused) while connecting to upstream, client: 172.31.45.215, server: , request: "GET /?a=fetch&content=<php>die(@md5(HelloThinkCMF))</php> HTTP/1.1", upstream: "http://127.0.0.1:8000/?a=fetch&content=<php>die(@md5(HelloThinkCMF))</php>", host: "44.237.64.108"
2021/07/26 05:18:07 [error] 6904#6904: *1802 connect() failed (111: Connection refused) while connecting to upstream, client: 172.31.45.215, server: , request: "GET /solr/admin/info/system?wt=json HTTP/1.1", upstream: "http://127.0.0.1:8000/solr/admin/info/system?wt=json", host: "44.237.64.108"
2021/07/26 05:18:07 [error] 6904#6904: *1802 connect() failed (111: Connection refused) while connecting to upstream, client: 172.31.45.215, server: , request: "GET /index.php?s=/Index/\think\app/invokefunction&function=call_user_func_array&vars[0]=md5&vars[1][]=HelloThinkPHP21 HTTP/1.1", upstream: "http://127.0.0.1:8000/index.php?s=/Index/\think\app/invokefunction&function=call_user_func_array&vars[0]=md5&vars[1][]=HelloThinkPHP21", host: "44.237.64.108"
2021/07/26 05:18:08 [error] 6904#6904: *1802 connect() failed (111: Connection refused) while connecting to upstream, client: 172.31.45.215, server: , request: "GET /?XDEBUG_SESSION_START=phpstorm HTTP/1.1", upstream: "http://127.0.0.1:8000/?XDEBUG_SESSION_START=phpstorm", host: "44.237.64.108"
2021/07/26 05:18:08 [error] 6904#6904: *1802 connect() failed (111: Connection refused) while connecting to upstream, client: 172.31.45.215, server: , request: "GET /wp-content/plugins/wp-file-manager/readme.txt HTTP/1.1", upstream: "http://127.0.0.1:8000/wp-content/plugins/wp-file-manager/readme.txt", host: "44.237.64.108"
2021/07/26 05:18:08 [error] 6904#6904: *1802 connect() failed (111: Connection refused) while connecting to upstream, client: 172.31.45.215, server: , request: "GET /console/ HTTP/1.1", upstream: "http://127.0.0.1:8000/console/", host: "44.237.64.108"
2021/07/26 05:18:09 [error] 6904#6904: *1802 connect() failed (111: Connection refused) while connecting to upstream, client: 172.31.45.215, server: , request: "POST /Autodiscover/Autodiscover.xml HTTP/1.1", upstream: "http://127.0.0.1:8000/Autodiscover/Autodiscover.xml", host: "44.237.64.108"
2021/07/26 05:18:09 [error] 6904#6904: *1802 connect() failed (111: Connection refused) while connecting to upstream, client: 172.31.45.215, server: , request: "GET /_ignition/execute-solution HTTP/1.1", upstream: "http://127.0.0.1:8000/_ignition/execute-solution", host: "44.237.64.108"
2021/07/26 05:18:42 [error] 6904#6904: *1802 connect() failed (111: Connection refused) while connecting to upstream, client: 172.31.45.215, server: , request: "GET /manager/html HTTP/1.1", upstream: "http://127.0.0.1:8000/manager/html", host: "44.237.64.108"
2021/07/26 05:22:50 [error] 8765#8765: *44 connect() failed (111: Connection refused) while connecting to upstream, client: 172.31.15.153, server: , request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:8000/", host: "djangoproject.eba-fuipjhfa.us-west-2.elasticbeanstalk.com"
2021/07/26 05:22:51 [error] 8765#8765: *44 connect() failed (111: Connection refused) while connecting to upstream, client: 172.31.15.153, server: , request: "GET /favicon.ico HTTP/1.1", upstream: "http://127.0.0.1:8000/favicon.ico", host: "djangoproject.eba-fuipjhfa.us-west-2.elasticbeanstalk.com", referrer: "http://djangoproject.eba-fuipjhfa.us-west-2.elasticbeanstalk.com/"
2021/07/26 05:24:18 [error] 8765#8765: *68 connect() failed (111: Connection refused) while connecting to upstream, client: 172.31.15.153, server: , request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:8000/", host: "djangoproject.eba-fuipjhfa.us-west-2.elasticbeanstalk.com"
2021/07/26 05:24:18 [error] 8765#8765: *68 connect() failed (111: Connection refused) while connecting to upstream, client: 172.31.15.153, server: , request: "GET /favicon.ico HTTP/1.1", upstream: "http://127.0.0.1:8000/favicon.ico", host: "djangoproject.eba-fuipjhfa.us-west-2.elasticbeanstalk.com", referrer: "http://djangoproject.eba-fuipjhfa.us-west-2.elasticbeanstalk.com/"
2021/07/26 05:37:38 [error] 9377#9377: *72 connect() failed (111: Connection refused) while connecting to upstream, client: 172.31.15.153, server: , request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:8000/", host: "djangoproject.eba-fuipjhfa.us-west-2.elasticbeanstalk.com"
2021/07/26 05:37:39 [error] 9377#9377: *72 connect() failed (111: Connection refused) while connecting to upstream, client: 172.31.15.153, server: , request: "GET /favicon.ico HTTP/1.1", upstream: "http://127.0.0.1:8000/favicon.ico", host: "djangoproject.eba-fuipjhfa.us-west-2.elasticbeanstalk.com", referrer: "http://djangoproject.eba-fuipjhfa.us-west-2.elasticbeanstalk.com/"

From the looks of it, the deployment seems successful. However, based on a handful of other posts like this, the nginx logs seem to show there may have been something wrong with getting gunicorn to work as the web server gateway interface. Any advice on how to solve this problem?

Answer 1

my deployment is still not good for production as I still have to provision manually from aws console the postgres database and change the SG inbound rules for the database.

so far this is what I came up with.

this is my diango.config file inside .ebextensions/ :

option_settings:
  aws:elasticbeanstalk:application:environment:
    DJANGO_SETTINGS_MODULE: "projectname.settings"
    PYTHONPATH: "/var/app/current:$PYTHONPATH"
    AWS_STORAGE_BUCKET_NAME: "nameofbucket"
    AWS_S3_REGION_NAME: "yourregion"
    RDS_USERNAME: "dbusername"
    RDS_PASSWORD: "dbpassword"
    RDS_PORT: 5432
  aws:elasticbeanstalk:container:python:
    WSGIPath: "projectname.wsgi:application"
    NumProcesses: 3
    NumThreads: 20
  aws:elasticbeanstalk:environment:process:default:
    HealthCheckPath: "/about"
    MatcherHTTPCode: "200-499"
  aws:elasticbeanstalk:environment:proxy:staticfiles:
    /static: static
    /media: media
container_commands:
  01_makemigrations:
    command: "source /var/app/venv/*/bin/activate && python3 manage.py makemigrations --noinput"
    leader_only: true
  02_migrate:
    command: "source /var/app/venv/*/bin/activate && python3 manage.py migrate --noinput"
    leader_only: true
  03_createsu:
    command: "source /var/app/venv/*/bin/activate && python3 manage.py createsu"
  04_collectstatic:
    command: "source /var/app/venv/*/bin/activate && python3 manage.py collectstatic --noinput"
    leader_only: true

this will create an enviroment that has not a database, I then provision from the console a postgres database.

I then run from terminal a script to set enviromental variable inside elastic beanstalk:

 #!/bin/bash
eb setenv \
AWS_ACCESS_KEY_ID='' \
AWS_SECRET_ACCESS_KEY='' \
RDS_DB_NAME='' \
RDS_HOSTNAME=''

inside my setting.py file I pass something like this:

from decouple import config
DEFAULT_AUTO_FIELD = "django.db.models.BigAutoField"

STATIC_URL = "/static/"
STATIC_ROOT = BASE_DIR / "static"

MEDIA_URL = "/media/"
MEDIA_ROOT = BASE_DIR / "media"


try:
    from .local_settings import *

    AWS_ACCESS_KEY_ID = config("AWS_ACCESS_KEY_ID")
    AWS_SECRET_ACCESS_KEY = config("AWS_SECRET_ACCESS_KEY")
    AWS_STORAGE_BUCKET_NAME = config("AWS_STORAGE_BUCKET_NAME")
    AWS_S3_REGION_NAME = config("AWS_S3_REGION_NAME")

except ImportError:
    print("no local file found")

    ## Django storages - use this for production
    STATICFILES_STORAGE = "storages.backends.s3boto3.S3Boto3Storage"
    DEFAULT_FILE_STORAGE = "storages.backends.s3boto3.S3Boto3Storage"
    print("no local file found you are in production")
    # Database
    # https://docs.djangoproject.com/en/3.2/ref/settings/#databases

    DATABASES = {
        "default": {
            "ENGINE": "django.db.backends.postgresql_psycopg2",
            "NAME": os.environ["RDS_DB_NAME"],
            "USER": os.environ["RDS_USERNAME"],
            "PASSWORD": os.environ["RDS_PASSWORD"],
            "HOST": os.environ["RDS_HOSTNAME"],
            "PORT": os.environ["RDS_PORT"],
        }
    }
    AWS_ACCESS_KEY_ID = config("AWS_ACCESS_KEY_ID")
    AWS_SECRET_ACCESS_KEY = config("AWS_SECRET_ACCESS_KEY")
    AWS_STORAGE_BUCKET_NAME = config("AWS_STORAGE_BUCKET_NAME")
    AWS_S3_REGION_NAME = config("AWS_S3_REGION_NAME")

this makes my project up and running on Elastic Beanstalk. as I said it is still not fully automatic as the part where I provision the database inside the config file does not work. Hope it is useful.

Answer 2

It seems like port 80 is important for AWS Elastic Beanstalk running a Docker environment with nginx. Make sure you EXPOSE 80 and EXPOSE 8000 in your Dockerfile . I was using docker-compose too, so I had to add ports: - 80:8000 to my docker-compose.yml .