nginx, gunicorn and django timing out

Question

I'm so confused! I set everything up, my site was working for two days, and then suddenly today it stops working. The only thing I changed was yesterday I was trying to serve PHP files so I installed PHP and uwsgi. It was late and I didn't realize what I was doing. It was from this website: http://uwsgi-docs.readthedocs.org/en/latest/PHP.html

# Add ppa with libphp5-embed package
sudo add-apt-repository ppa:l-mierzwa/lucid-php5
# Update to use package from ppa
sudo apt-get update
# Install needed dependencies
sudo apt-get install php5-dev libphp5-embed libonig-dev libqdbm-dev
# Compile uWSGI PHP plugin
python uwsgiconfig --plugin plugins/php

But didn't change any settings. Even after doing that, everything was still fine. However the next day, my site just doesn't load.

I tried a few things which didn't work. In my settings:

ALLOWED_HOSTS = ['*']

In my gunicorn.sh, I set TIMEOUT=60. However, when I try to access my site (lewischi.com), nothing even happens. But when I go to http://127.0.0.1:8000 , I do see workers doing stuff and get a 404 error.

Using the URLconf defined in django_project.urls, 
Django tried these URL patterns, in this order: 

I'm not sure what's going on! nginx-error log isn't very helpful but the access log seems more useful.

From my nginx-access.log (it works, then stops working):

50.156.86.221 - - [25/Sep/2015:00:25:43 -0700] "GET /codeWindow.html 
HTTP/1.1" 200 2081 "http://lewischi.com/" "Mozilla/5.0 (Windows NT 6.1; WOW64) 
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/44.0.2403.157 Safari/537.36"
50.156.86.221 - - [25/Sep/2015:00:25:58 -0700] "GET /test.jpg HTTP/1.1" 
404 208 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 
(KHTML, like Gecko) Chrome/44.0.2403.157 Safari/537.36"
192.168.2.6 - - [25/Sep/2015:16:42:19 -0700] "GET / HTTP/1.1" 200 9596 "-" "-"
192.168.2.6 - - [25/Sep/2015:17:24:44 -0700] "GET / HTTP/1.1" 200 9596 "-" "-"
192.168.2.6 - - [25/Sep/2015:23:28:51 -0700] "GET / HTTP/1.1" 200 9596 "-" "-"
192.168.2.6 - - [25/Sep/2015:23:29:02 -0700] "GET / HTTP/1.1" 200 9596 "-" "-"

From my supervisor log file:

supervisor: couldn't exec /home/lewischi/projects/active/django_project/gunicorn.sh: ENOEXEC   
supervisor: child process was not spawned

ANY HELP would be greatly appreciated!!!! I feel like I should just uninstall uwsgi. I don't want to break anything so I'm asking for advice before I go messing things up.

I'm pretty new to this so I may be overlooking something obvious. My gunicorn debug mode output:

    “Starting ”djangotut” as lewischi”
[2015-09-26 17:50:28 +0000] [2316] [DEBUG] Current configuration:
  proxy_protocol: False
  worker_connections: 1000
  statsd_host: None
  max_requests_jitter: 0
  post_fork: <function post_fork at 0x7faf049ec848>
  pythonpath: None
  enable_stdio_inheritance: False
  worker_class: sync
  ssl_version: 3
  suppress_ragged_eofs: True
  syslog: False
  syslog_facility: user
  when_ready: <function when_ready at 0x7faf049ec578>
  pre_fork: <function pre_fork at 0x7faf049ec6e0>
  cert_reqs: 0
  preload_app: False
  keepalive: 2
  accesslog: None
  group: 1000
  graceful_timeout: 30
  do_handshake_on_connect: False
  spew: False
  workers: 3
  proc_name: ”djangotut”
  sendfile: True
  pidfile: None
  umask: 0
  on_reload: <function on_reload at 0x7faf049ec410>
  pre_exec: <function pre_exec at 0x7faf049ecde8>
  worker_tmp_dir: None
  post_worker_init: <function post_worker_init at 0x7faf049ec9b0>
  limit_request_fields: 100
  on_exit: <function on_exit at 0x7faf049f2500>
  config: None
  secure_scheme_headers: {'X-FORWARDED-PROTOCOL': 'ssl', 'X-FORWARDED-PROTO': 'https', 'X-FORWARDED-SSL': 'on'}
  proxy_allow_ips: ['127.0.0.1']
  pre_request: <function pre_request at 0x7faf049ecf50>
  post_request: <function post_request at 0x7faf049f20c8>
  user: 1000
  forwarded_allow_ips: ['127.0.0.1']
  worker_int: <function worker_int at 0x7faf049ecb18>
  threads: 1
  max_requests: 1
  limit_request_line: 4094
  access_log_format: %(h)s %(l)s %(u)s %(t)s "%(r)s" %(s)s %(b)s "%(f)s" "%(a)s"
  certfile: None
  worker_exit: <function worker_exit at 0x7faf049f2230>
  chdir: /home/lewischi/projects/active/django_project
  paste: None
  default_proc_name: django_project.wsgi:application
  errorlog: -
  loglevel: DEBUG
  logconfig: None
  syslog_addr: udp://localhost:514
  syslog_prefix: None
  daemon: False
  ciphers: TLSv1
  on_starting: <function on_starting at 0x7faf049ec2a8>
  worker_abort: <function worker_abort at 0x7faf049ecc80>
  bind: ['0.0.0.0:8000']
  raw_env: []
  reload: False
  check_config: False
  limit_request_field_size: 8190
  nworkers_changed: <function nworkers_changed at 0x7faf049f2398>
  timeout: 60
  ca_certs: None
  django_settings: None
  tmp_upload_dir: None
  keyfile: None
  backlog: 2048
  logger_class: gunicorn.glogging.Logger
  statsd_prefix: 
[2015-09-26 17:50:28 +0000] [2316] [INFO] Starting gunicorn 19.3.0
[2015-09-26 17:50:28 +0000] [2316] [DEBUG] Arbiter booted
[2015-09-26 17:50:28 +0000] [2316] [INFO] Listening at: http://0.0.0.0:8000 (2316)
[2015-09-26 17:50:28 +0000] [2316] [INFO] Using worker: sync
[2015-09-26 17:50:28 +0000] [2327] [INFO] Booting worker with pid: 2327
[2015-09-26 17:50:28 +0000] [2328] [INFO] Booting worker with pid: 2328
[2015-09-26 17:50:28 +0000] [2329] [INFO] Booting worker with pid: 2329
[2015-09-26 17:50:29 +0000] [2316] [DEBUG] 3 workers
[2015-09-26 17:50:30 +0000] [2316] [DEBUG] 3 workers

Answer 1

The problem is not with supervisord itself, few things to consider when dealing with Nginx, Gunicorn and Django in general:

1. Make sure the user running the app process(minimum 1 user non root not including users created by default for eg: Nginx, Postgresql. Changes with the stack) has the right permissions and ownership to achieve it's goals.
2. When adding another app to your stack, you should first check the port it runs on by default, and change it to prevent port conflicts, keep in mind the difference between internal and external ports since you use Nginx as a proxy to Gunicorn( this is what causes most timeouts , happened to me several times at late night work), you can use Nginx as a proxy server and create many apps with different unique internal port for each app.

With the error log you provided for supervisor, it seems you're running your gunicorn.sh either with a user that doesn't have enough permissions or ownership, or executing with a wrong command.

Please provide the supervisor config file relevant to your app.

Update : seems his ip address changed.

Answer 2

Ah never mind. Thanks for your time. It turned out that my ip address somehow changed which should not have happened.... Rookie mistake.