Import error when starting uwsgi with virtual environments

Question

I am trying to setup my Django application to work with python 3.5 ( Python 2.7 is default)

So I created a virtual environment in 3.5 and deployed my Django project.

I now want to get it working with uwsgi

/etc/uwsgi/emperor.ini

[uwsgi]
emperor = etc/uwsgi/sites
logto = /var/log/uwsgi

/etc/uwsgi/sites/ams.ini

[uwsgi]
project = AMS
base = /home/user

chdir = %(base)/%(project)
#home = %(base)/.virtualenvs/%(project)
module = %(project).wsgi:application

master = true
processes = 5
socket = %(base)/%(project)/ams.sock
chmod-socket = 666
uid = user
gid = www-data
vacuum = true
logto = /var/log/uwsgi/log.log

/etc/systemd/system/uwsgi.service

[Unit]
Description=uWSGI Emperor service
After=syslog.target

[Service]
ExecStart=/usr/local/bin/uwsgi --http :8000 --emperor /etc/uwsgi/sites
Restart=always
KillSignal=SIGQUIT

Type=notify
StandardError=syslog
NotifyAccess=all

[Install]
WantedBy=multi-user.target

If I start uwsgi from within the virtual environment I can start it successfully

uwsgi --http :8080 --chdir /home/user/AMS -w AMS.wsgi 
[pid: 31757|app: 0|req: 2/2] 203.94.69.162 () {42 vars in 695 bytes} [Mon Apr 17 10:34:02 2017] GET /ams_tools/ => generated 146 bytes in 76 msecs (HTTP/1.1 200) 2 headers in 80 bytes (1 switches on core 0)

However when I start uswgi as a service I get the following error

*** Starting uWSGI 2.0.15 (64bit) on [Tue Apr 18 07:33:41 2017] ***
compiled with version: 5.4.0 20160609 on 17 April 2017 07:06:40
os: Linux-4.8.0-45-generic #48~16.04.1-Ubuntu SMP Fri Mar 24 12:46:56 UTC 2017
nodename: ams-de1
machine: x86_64
clock source: unix
detected number of CPU cores: 8
current working directory: /etc/uwsgi/sites
detected binary path: /usr/local/bin/uwsgi
!!! no internal routing support, rebuild with pcre support !!!
setgid() to 33
set additional group 1002 (admin)
setuid() to 1002
chdir() to /home/dhanushka/CxenseAMS
your processes number limit is 256882
your memory page size is 4096 bytes
detected max file descriptor number: 1024
lock engine: pthread robust mutexes
thunder lock: disabled (you can enable it with --thunder-lock)
uwsgi socket 0 bound to UNIX address /home/dhanushka/CxenseAMS/ams.sock fd 3
Python version: 2.7.12 (default, Nov 19 2016, 06:48:10)  [GCC 5.4.0 20160609]
*** Python threads support is disabled. You can enable it with --enable-threads ***
Python main interpreter initialized at 0xf3fd70
your server socket listen backlog is limited to 100 connections
your mercy for graceful operations on workers is 60 seconds
mapped 436560 bytes (426 KB) for 5 cores
*** Operational MODE: preforking ***
Traceback (most recent call last):
  File "./CxenseAMS/wsgi.py", line 19, in <module>
    application = get_wsgi_application()
  File "/home/dhanushka/.virtualenvs/amsAPI/lib/python3.5/site-packages/django/core/wsgi.py", line 13, in get_wsgi_application
    django.setup(set_prefix=False)
  File "/home/dhanushka/.virtualenvs/amsAPI/lib/python3.5/site-packages/django/__init__.py", line 27, in setup
    apps.populate(settings.INSTALLED_APPS)
  File "/home/dhanushka/.virtualenvs/amsAPI/lib/python3.5/site-packages/django/apps/registry.py", line 108, in populate
    app_config.import_models(all_models)
  File "/home/dhanushka/.virtualenvs/amsAPI/lib/python3.5/site-packages/django/apps/config.py", line 199, in import_models
    self.models_module = import_module(models_module_name)
  File "/usr/lib/python2.7/importlib/__init__.py", line 37, in import_module
    __import__(name)
  File "./CxenseAMS/models.py", line 5, in <module>
    class SegmentTrafficAbstract(models.Model):
  File "/home/dhanushka/.virtualenvs/amsAPI/lib/python3.5/site-packages/django/db/models/base.py", line 119, in __new__
    new_class.add_to_class('_meta', Options(meta, app_label))
  File "/home/dhanushka/.virtualenvs/amsAPI/lib/python3.5/site-packages/django/db/models/base.py", line 316, in add_to_class
    value.contribute_to_class(cls, name)
  File "/home/dhanushka/.virtualenvs/amsAPI/lib/python3.5/site-packages/django/db/models/options.py", line 214, in contribute_to_class
    self.db_table = truncate_name(self.db_table, connection.ops.max_name_length())
  File "/home/dhanushka/.virtualenvs/amsAPI/lib/python3.5/site-packages/django/db/__init__.py", line 33, in __getattr__
    return getattr(connections[DEFAULT_DB_ALIAS], item)
  File "/home/dhanushka/.virtualenvs/amsAPI/lib/python3.5/site-packages/django/db/utils.py", line 211, in __getitem__
    backend = load_backend(db['ENGINE'])
  File "/home/dhanushka/.virtualenvs/amsAPI/lib/python3.5/site-packages/django/db/utils.py", line 115, in load_backend
    return import_module('%s.base' % backend_name)
  File "/usr/lib/python2.7/importlib/__init__.py", line 37, in import_module
    __import__(name)
  File "/home/dhanushka/.virtualenvs/amsAPI/lib/python3.5/site-packages/django/db/backends/postgresql/base.py", line 24, in <module>
    raise ImproperlyConfigured("Error loading psycopg2 module: %s" % e)
django.core.exceptions.ImproperlyConfigured: Error loading psycopg2 module: No module named _psycopg
unable to load app 0 (mountpoint='') (callable not found or import error)
*** no app loaded. going in full dynamic mode ***
*** uWSGI is running in multiple interpreter mode ***
spawned uWSGI master process (pid: 3534)
spawned uWSGI worker 1 (pid: 3537, cores: 1)
spawned uWSGI worker 2 (pid: 3538, cores: 1)
spawned uWSGI worker 3 (pid: 3539, cores: 1)
spawned uWSGI worker 4 (pid: 3540, cores: 1)
spawned uWSGI worker 5 (pid: 3541, cores: 1)

I can see that it is not using the correct python version. I tried to set the home location to /home/dhanushka/.virtualenvs/amsAPI but then I get

ImportError: No module named site

Answer 1

You need to tell your uWSGI the path to venv .

http://uwsgi-docs.readthedocs.io/en/latest/Options.html#virtualenv

Answer 2

The path in pythonpath parameter must be to the python3 interpreter, inside virtualenv:

/[absolute path to env]/bin/python3