Django can't find locally-installed Python modules

Question

I'm running a Django-based website that's hosted on a shared server (ie I have only a limited account, no admin privileges). The server's administrator installed the Django modules for Python 2.7, but now I want to add a 3rd-party Django library I found called django-bleach. So I installed it locally, using

pip install --user django-bleach

Then, following the instructions in the django-bleach documentation , I added 'django-bleach' to my INSTALLED_APPS list in settings.py . However, when I touched wsgi.py to get Apache to reload my code, I got an ImportError in my Apache log. Usually this means that the module I'm trying to import isn't in the PYTHONPATH, so I put a print sys.path statement in wsgi.py . This is what happens:

[Mon Nov 24 13:44:58 2014] [error] sys.path is: ['/extra/www/html/quotes/quotes_django', '/opt/rh/python27/root/usr/lib64/python2.7/site-packages', 
'/usr/lib64/python27.zip', '/usr/lib64/python2.7', '/usr/lib64/python2.7/plat-linux2', '/usr/lib64/python2.7/lib-tk', '/usr/lib64/python2.7/lib-old', 
'/usr/lib64/python2.7/lib-dynload', '/usr/lib64/python2.7/site-packages', '/usr/lib/python2.7/site-packages', '/users/ejt64/.local/lib/python2.7/site-packages',
'/users/ejt64/.local/lib/python2.7/site-packages', '/users/ejt64/.local/lib/python2.7/site-packages']
[Mon Nov 24 13:44:58 2014] [error] mod_wsgi (pid=29372): Target WSGI script '/extra/www/html/quotes/quotes_django/quotes_django/wsgi.py' cannot be loaded as Python module.
[Mon Nov 24 13:44:58 2014] [error] mod_wsgi (pid=29372): Exception occurred processing WSGI script '/extra/www/html/quotes/quotes_django/quotes_django/wsgi.py'.
[Mon Nov 24 13:44:58 2014] [error] Traceback (most recent call last):
[Mon Nov 24 13:44:58 2014] [error]   File "/extra/www/html/quotes/quotes_django/quotes_django/wsgi.py", line 17, in <module>
[Mon Nov 24 13:44:58 2014] [error]     application = get_wsgi_application()
[Mon Nov 24 13:44:58 2014] [error]   File "/opt/rh/python27/root/usr/lib64/python2.7/site-packages/django/core/wsgi.py", line 14, in get_wsgi_application
[Mon Nov 24 13:44:58 2014] [error]     django.setup()
[Mon Nov 24 13:44:58 2014] [error]   File "/opt/rh/python27/root/usr/lib64/python2.7/site-packages/django/__init__.py", line 21, in setup
[Mon Nov 24 13:44:58 2014] [error]     apps.populate(settings.INSTALLED_APPS)
[Mon Nov 24 13:44:58 2014] [error]   File "/opt/rh/python27/root/usr/lib64/python2.7/site-packages/django/apps/registry.py", line 85, in populate
[Mon Nov 24 13:44:58 2014] [error]     app_config = AppConfig.create(entry)
[Mon Nov 24 13:44:58 2014] [error]   File "/opt/rh/python27/root/usr/lib64/python2.7/site-packages/django/apps/config.py", line 87, in create
[Mon Nov 24 13:44:58 2014] [error]     module = import_module(entry)
[Mon Nov 24 13:44:58 2014] [error]   File "/usr/lib64/python2.7/importlib/__init__.py", line 37, in import_module
[Mon Nov 24 13:44:58 2014] [error]     __import__(name)
[Mon Nov 24 13:44:58 2014] [error] ImportError: No module named django_bleach

Note that the /users/ejt64/.local/lib/python2.7/site-packages/ directory, which appears three times in sys.path, is my user-local Python location where django-bleach is installed. I made this directory world-readable, in case the Apache process was running into a permissions problem accessing my directory. So how can I be getting an ImportError?

It must be something to do with WSGI/Django, though, because if I just run Python in my Django app directory, I can do this:

Python 2.7.5 (default, Dec  3 2013, 08:35:16) 
[GCC 4.4.6 20120305 (Red Hat 4.4.6-4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import django_bleach
>>> 

But my Django application somehow can't find the django_bleach module, even though it's looking in the right location.

Additional Info : My wsgi.py is just the standard one generated by django-admin startproject , plus a print statement:

import os 
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "quotes_django.settings")

import sys
print >>sys.stderr, "sys.path is: " + str(sys.path)

from django.core.wsgi import get_wsgi_application
application = get_wsgi_application()

Answer 1

Well, I finally solved my problem. It turns out that although /users/ejt64/.local/lib/python2.7/site-packages/ was on the PYTHONPATH being searched by WSGI, I had forgotten to make one of the directories along that path world- executable , so the Apache user couldn't access it.

Although all the directories and files were world- readable , a user can't read a world-readable file unless it has execute permissions on all the directories that contain it, and it turns out my home directory ( /users/ejt64 ) was world-readable but not world-executable. Once I made this directory world-executable, the ImportError went away.

It sure would be nice if I could make Apache/WSGI/Python print an error message when it tries to read a directory on the PYTHONPATH but can't access it due to permissions.

Answer 2

~/.local/lib/python2.7/site-packages/

Is relative to the user running apache. What user is running apache? Typically apache runs as the default user 'nobody' or 'apache'. Having a relative path to '~' will point to the home directory of the user running apache.

I'd recommend using a virtualenv or specifying the absolute path to your local pip packages. Going into your site-packages directory and running pwd will let you know the absolute path. You may need to alter the permissions of your pip packages and directories as well if the user running apache can't read/write/execute against them.

It works fine when you drop into the Python REPL because you are executing everything as your user and '~' points to your home directory.

If you have access to edit the apache config files on your server, I'd recommend taking a look at this Django documentation, https://docs.djangoproject.com/en/dev/howto/deployment/wsgi/modwsgi/#using-a-virtualenv Event if you don't use a virtualenv you will still need to set this accordingly to the absolute path of your local site-packages directory.

WSGIPythonPath /path/to/mysite.com:/path/to/your/venv/lib/python3.X/site-packages