Gunicorn Supervisor Startup Error

Question

I've followed this tutorial twice, but on the second machine that I've run it on I get a supervisor-run gunicorn error. When I tell supervisor to startup gunicorn using:

$ sudo supervisorctl start gunicorn
gunicorn: ERROR (abnormal termination)

The gunicorn_err.log repeats this:

Unknown command: 'run_gunicorn'
Type 'manage.py help' for usage.

The supervisor config looks like:

[program:gunicorn]
command=/home/ubuntu/.virtualenvs/<VIRTUALENV>/bin/python /home/ubuntu/<APPNAME>/manage.py run_gunicorn -w 4 -k gevent
directory=/home/ubuntu/<APPNAME>
user=www-data
autostart=true
autorestart=true
stdout_logfile = /var/log/supervisor/gunicorn.log
stderr_logfile = /var/log/supervisor/gunicorn_err.log

The gunicorn.log is empty. I've tried changing the user to ubuntu and running without a virtualenv (my 'default' python environment is also ready to go as in it has all pre-requisite packages.) I have even tried removing spaces between the variable assignments in gunicorn.conf Actually if I manually run:

$ /usr/bin/python /home/ubuntu/<APPNAME>/manage.py run_gunicorn -w 4 -k gevent
2013-01-22 19:20:33 [1231] [INFO] Starting gunicorn 0.17.2
2013-01-22 19:20:33 [1231] [INFO] Listening at: http://127.0.0.1:8000 (1231)
2013-01-22 19:20:33 [1231] [INFO] Using worker: gevent
2013-01-22 19:20:33 [1236] [INFO] Booting worker with pid: 1236
2013-01-22 19:20:33 [1237] [INFO] Booting worker with pid: 1237
2013-01-22 19:20:33 [1238] [INFO] Booting worker with pid: 1238
2013-01-22 19:20:33 [1239] [INFO] Booting worker with pid: 1239

And same with the virtualenv python run:

$ /home/ubuntu/.virtualenvs/<VIRTUALENV>/bin/python /home/ubuntu/<APPNAME>/manage.py run_gunicorn -w 4 -k gevent
2013-01-22 19:21:53 [1246] [INFO] Starting gunicorn 0.17.2
2013-01-22 19:21:53 [1246] [INFO] Listening at: http://127.0.0.1:8000 (1246)
2013-01-22 19:21:53 [1246] [INFO] Using worker: gevent
2013-01-22 19:21:53 [1251] [INFO] Booting worker with pid: 1251
2013-01-22 19:21:53 [1252] [INFO] Booting worker with pid: 1252
2013-01-22 19:21:53 [1253] [INFO] Booting worker with pid: 1253
2013-01-22 19:21:53 [1254] [INFO] Booting worker with pid: 1254

How is it possible that the supervisor initiated gunicorn is unable to find the 'run_gunicorn' command when I can run it using any python environment and it works? And yes 'gunicorn' , is in the INSTALLED_APPS

Answer 1

Turns out it was an environmental variable that I was getting in settings.py that didn't exist under supervisord's start gunicorn.

In addition, the environmental variable error was being suppressed and never made it to the gunicorn_err.log

When I switched the gunicorn.conf command to :

command = /usr/local/bin/gunicorn_django -w 4 -k gevent

I could see a clearer error in gunicorn_err.log

    raise KeyError(key)
KeyError: 'AWS_STORAGE_BUCKET_NAME'
2013-01-22 22:51:09 [2290] [INFO] Worker exiting (pid: 2290)

To address this I just didn't use an environmental variable got the variable using a different means. It worked then an I reverted back to the original virtualenv run_gunicorn command

command=/home/ubuntu/.virtualenvs/<VIRTUALENV>/bin/python /home/ubuntu/<APPNAME>/manage.py run_gunicorn -w 4 -k gevent

If continuing to use environmental variables in settings is important, take a look at supervisord-configuration there seems to be a way to configure key/value environment variable pairs for supervisord executed applications.