Installing psycopg2 in a virtualenv in ubuntu os

Question

I using a virtualenv and I am trying to set up postgresql. I have psycopg2 installed on my system, but it appears that my virtual env does not recognize psycopg2. Is it possible to just copy the system wide psycopg2 files into my django project directory or is this a bad approach to take?

This is the relevant part of my settings.py file in my django project.

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.postgresql_psycopg2',
        'NAME': 'django_db' ,
    'USER': 'myusername' ,
    'PASSWORD': 'mypassword' ,
    'HOST': 'localhost' ,
    'PORT': '' ,

    }
}

When I run python manage.py migrate I get this error:

    File "manage.py", line 10, in <module>
        execute_from_command_line(sys.argv)
      File "/uwsgi-tutorial/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 385, in execute_from_command_line
        utility.execute()
      File "/uwsgi-tutorial/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 354, in execute
        django.setup()
      File "/uwsgi-tutorial/local/lib/python2.7/site-packages/django/__init__.py", line 21, in setup
        apps.populate(settings.INSTALLED_APPS)
      File "/uwsgi-tutorial/local/lib/python2.7/site-packages/django/apps/registry.py", line 108, in populate
        app_config.import_models(all_models)
      File "/uwsgi-tutorial/local/lib/python2.7/site-packages/django/apps/config.py", line 197, in import_models
        self.models_module = import_module(models_module_name)
      File "/usr/lib/python2.7/importlib/__init__.py", line 37, in import_module
        __import__(name)
      File "/uwsgi-tutorial/local/lib/python2.7/site-packages/django/contrib/auth/models.py", line 40, in <module>
        class Permission(models.Model):
      File "/uwsgi-tutorial/local/lib/python2.7/site-packages/django/db/models/base.py", line 125, in __new__
        new_class.add_to_class('_meta', Options(meta, **kwargs))
      File "/uwsgi-tutorial/local/lib/python2.7/site-packages/django/db/models/base.py", line 300, in add_to_class
        value.contribute_to_class(cls, name)
      File "/uwsgi-tutorial/local/lib/python2.7/site-packages/django/db/models/options.py", line 166, in contribute_to_class
        self.db_table = truncate_name(self.db_table, connection.ops.max_name_length())
      File "/uwsgi-tutorial/local/lib/python2.7/site-packages/django/db/__init__.py", line 40, in __getattr__
        return getattr(connections[DEFAULT_DB_ALIAS], item)
      File "/uwsgi-tutorial/local/lib/python2.7/site-packages/django/db/utils.py", line 242, in __getitem__
        backend = load_backend(db['ENGINE'])
      File "/uwsgi-tutorial/local/lib/python2.7/site-packages/django/db/utils.py", line 108, in load_backend
        return import_module('%s.base' % backend_name)
      File "/usr/lib/python2.7/importlib/__init__.py", line 37, in import_module
        __import__(name)
      File "/uwsgi-tutorial/local/lib/python2.7/site-packages/django/db/backends/postgresql_psycopg2/base.py", line 27, in <module>
        raise ImproperlyConfigured("Error loading psycopg2 module: %s" % e)
    django.core.exceptions.ImproperlyConfigured: Error loading psycopg2 module: No module named psycopg2

When I do pip install psycopg2 in my virtual env, I get the following error:

Exception:
Traceback (most recent call last):
  File "/uwsgi-tutorial/local/lib/python2.7/site-packages/pip/basecommand.py", line 122, in main
    status = self.run(options, args)
  File "/uwsgi-tutorial/local/lib/python2.7/site-packages/pip/commands/install.py", line 278, in run
    requirement_set.prepare_files(finder, force_root_egg_info=self.bundle, bundle=self.bundle)
  File "/uwsgi-tutorial/local/lib/python2.7/site-packages/pip/req.py", line 1153, in prepare_files
    location = req_to_install.build_location(self.build_dir, not self.is_download)
  File "/uwsgi-tutorial/local/lib/python2.7/site-packages/pip/req.py", line 218, in build_location
    _make_build_dir(build_dir)
  File "/uwsgi-tutorial/local/lib/python2.7/site-packages/pip/req.py", line 1527, in _make_build_dir
    os.makedirs(build_dir)
  File "/uwsgi-tutorial/lib/python2.7/os.py", line 157, in makedirs
    mkdir(name, mode)
OSError: [Errno 13] Permission denied: '/uwsgi-tutorial/build'

Storing debug log for failure in /home/user/.pip/pip.log

Answer 1

This is an old thread but some people might experience Python.h not found type of error while pip-installing psycopg2 to virtualenv on Ubuntu based systems.

Check your virtualenv python version by running python -V inside virtualenv.

Assuming it is Python 3.8.0, try installing libpython3.8-dev to your system (not virtualenv) by:

sudo apt-get install libpython3.8-dev

Probably similar approach with other versions.

That solved it for me.

Answer 2

Based on the combination of your question and one of your comments, it appears that when you ran sudo pip install psycopg2 , it was using the system install, as it referred to psycopg2 in /usr/lib/python2.7/dist-packages which is different from the environment from which you're trying to run Django .

Your permissions error, which is from the Django environment, seems likely to be due to not using sudo (among other things, some C compilation needs to take place, during the psycopg2 install, and so sudo is often required whereas it isn't with many other packages).

I would recommend you refer to the virtualenv install's pip binary explicitly when running the sudo command.

ie: sudo <virtualenv_root>/bin/pip install psycopg2

Answer 3

Something that worked for me (python 3.8 in virtualenv): Inside the virtual environment type:

pip install psycopg2-binary

Best.