Celery Supervisor Virtualenv找不到通过github安装的pip软件包

Question

Hello I am using supervisor with celery in order to run some tasks in a django web application. Celery seems to be having an issue finding/resolving, models from packages that have been installed directly via github.

http://pip.readthedocs.org/en/stable/reference/pip_install/#vcs-support

As that states the packages are installed/cloned into a <virtualenv>/src directory. I am wondering if I am missing something in my environment setup with supervisord.conf file such as the PYTHONPATH or something else to have these installed packages available. I am getting a

ValueError: Related model cannot be resolved

which does not seem to make sense as the application as a whole is working fine with these packages.

Versions

Celery - 3.1.18

Supervisor - 3.0b2

supervisord.conf

[program:celeryd]
command=/home/path/to/virtualenv/celery worker -A moi --loglevel=info
stdout_logfile=/var/log/celery/celeryd.log
stderr_logfile=/var/log/celery/celeryd.error.log
directory=/path/to/app

The virtualenv looks structure looks like this

virtualenv/
  - bin/
  - lib/
  - share/
  - src/github-installed-egg

Error

File "/home/ubuntu/moi/lib/python3.4/site-packages/django/db/models/fields/related.py", line 1600, in resolve_related_fields
raise ValueError('Related model %r cannot be resolved' % self.rel.to)
ValueError: Related model 'ndptc_core_courses.Course' cannot be resolved

The web application is running using nginx and uwsgi, and works fine . The virtualenv setting in the uwsgi configuration file makes this work. The model is a foreign key that is declared by name and not the model object itself. Models that are from packages installed via pypi work fine as well its just those installed in the src folder that are causing the issue.

described in django docs here

Answer 1

Not sure if this is your problem, but a common scenario is to have your app installed under a virtualenv and supervisor installed directly on the host.

Under this scenario, you need to make sure that the celery version your are running is the one installed in your virtualenv and not a version that might be installed in the host's root environment.

As an example, lets assume your virtualenv is located at /opt/virtualenvs/foo and your supervisor program is configured at /etc/supervisor.d/foo.conf .

[program:foo]
command=/opt/virtualenvs/foo/bin/celery -A celery.conf worker
...

Answer 2

You must "activate" virtualenv in supervisor config. To do this, add following line to Your app config:

environment=PATH="/opt/virtualenvs/foo/bin",VIRTUAL_ENV="/opt/virtualenvs/foo",PYTHONPATH="/opt/virtualenvs/foo/lib/python:/opt/virtualenvs/foo/lib/python/site-packages"