cache backend works on devserver but not mod_wsgi

Question

I am using a custom cache backend to wrap the built-in cache backends so that I can add the current site_id to all the cache_keys (this is useful for multi-site functionality with a single memcached instance)

unfortunately it works great on the django built-in devserver, but give a nasty error when I try to run it on the live server with mod_wsgi

Here is the traceback from my error log:

[Wed Feb 17 00:39:34 2010] [error] [client 127.0.0.1] mod_wsgi (pid=22933): Exception occurred processing WSGI script '/home/jiaaro/webapps/op_wsgi/myProject/deploy/myProject.wsgi'.
[Wed Feb 17 00:39:34 2010] [error] [client 127.0.0.1] Traceback (most recent call last):
[Wed Feb 17 00:39:34 2010] [error] [client 127.0.0.1]   File "/home/jiaaro/webapps/op_wsgi/lib/python2.5/django/core/handlers/wsgi.py", line 230, in __call__
[Wed Feb 17 00:39:34 2010] [error] [client 127.0.0.1]     self.load_middleware()
[Wed Feb 17 00:39:34 2010] [error] [client 127.0.0.1]   File "/home/jiaaro/webapps/op_wsgi/lib/python2.5/django/core/handlers/base.py", line 40, in load_middleware
[Wed Feb 17 00:39:34 2010] [error] [client 127.0.0.1]     mod = import_module(mw_module)
[Wed Feb 17 00:39:34 2010] [error] [client 127.0.0.1]   File "/home/jiaaro/webapps/op_wsgi/lib/python2.5/django/utils/importlib.py", line 35, in import_module
[Wed Feb 17 00:39:34 2010] [error] [client 127.0.0.1]     __import__(name)
[Wed Feb 17 00:39:34 2010] [error] [client 127.0.0.1]   File "/home/jiaaro/webapps/op_wsgi/myProject/apps/site_settings/__init__.py", line 6, in <module>
[Wed Feb 17 00:39:34 2010] [error] [client 127.0.0.1]     import cache_wrapper
[Wed Feb 17 00:39:34 2010] [error] [client 127.0.0.1]   File "/home/jiaaro/webapps/op_wsgi/myProject/apps/site_settings/cache_wrapper.py", line 4, in <module>
[Wed Feb 17 00:39:34 2010] [error] [client 127.0.0.1]     from django.core.cache.backends.base import BaseCache
[Wed Feb 17 00:39:34 2010] [error] [client 127.0.0.1]   File "/home/jiaaro/webapps/op_wsgi/lib/python2.5/django/core/cache/__init__.py", line 73, in <module>
[Wed Feb 17 00:39:34 2010] [error] [client 127.0.0.1]     cache = get_cache(settings.CACHE_BACKEND)
[Wed Feb 17 00:39:34 2010] [error] [client 127.0.0.1]   File "/home/jiaaro/webapps/op_wsgi/lib/python2.5/django/core/cache/__init__.py", line 68, in get_cache
[Wed Feb 17 00:39:34 2010] [error] [client 127.0.0.1]     return getattr(module, 'CacheClass')(host, params)
[Wed Feb 17 00:39:34 2010] [error] [client 127.0.0.1] AttributeError: 'module' object has no attribute 'CacheClass'

if I run the devserver on the same machine (the live server) it works fine... I am able to do this without trouble:

$ cd /home/jiaaro/webapps/op_wsgi/myProject
$ python2.5 manage.py runserver

and in a separate ssh session...

$ wget 127.0.0.1:8000

The pages are served correctly, using memcached, and the live database. is there something different about the way mod_wsgi imports modules that I should know about?

maybe something about the single-process, single-threaded nature of the devserver?

I have been struggling with this for days now, any help would be appreciated

Extra info:

• Webfaction shared hosting (centos)
• Apache 2
• Python 2.5
• Django 1.1.1
• mod_wsgi 2.5
• MySql 5.0

More Details: - I have set the cache backend (and it is working since you can see it importing the correct module in the traceback) - There is a class called "CacheClass" in the module:

site_settings/cache_wrapper.py :

from django.conf import settings
CACHE_BACKEND = getattr(settings, 'CUSTOM_CACHE_BACKEND')

from django.core.cache.backends.base import BaseCache

class CacheClass(BaseCache):
    from decorators import accept_site

    def __init__(self, *args):
        from django.core.cache import get_cache
        self.WRAPPED_CACHE = get_cache(CACHE_BACKEND)

    @accept_site
    def add(self, site, key, *args):
        return self.WRAPPED_CACHE.add(self._key(site, key),*args)

    @accept_site
    def get(self, site, key, *args):
        return self.WRAPPED_CACHE.get(self._key(site, key),*args)

    ... (all the rest of the wrapped methods)

    def _key(self, site, key):
        from exceptions import NoCurrentSite
        if not site:
            raise NoCurrentSite
        return "%s|%s" % (site.id, key)

THe accept_site decorator works in conjunction with some middleware to figure out the current site. Here they are:

decorators.py:

def accept_site(fn):
    def decorator(self, *args, **kwargs):
        site = kwargs.get('site', None)
        try: 
            del kwargs['site']
        except KeyError: 
            pass

        from .middleware import get_current_site
        site = site or get_current_site()

        if not site:
            raise NoCurrentSite("The current site is not available via thread locals, please specify a site with the 'site' keyword argument")

        return fn(self, site, *args, **kwargs)

    return decorator

and the middleware.py

try:
    from threading import local
except ImportError:
    from django.utils._threading_local import local

from django.conf import settings
from django.contrib.sites.models import Site

DEFAULT_SITE_ID = 1

_thread_locals = local()
def get_current_site():
    return getattr(_thread_locals, 'site', None)

def set_current_site(site):
    setattr(_thread_locals, 'site', site)

class SiteSettings(object):
    """Middleware that gets various objects from the
    request object and saves them in thread local storage."""
    def process_request(self, request):
        if settings.DEBUG:
            site_id = request.GET.get('site_id', DEFAULT_SITE_ID)
        else:
            site_id = DEFAULT_SITE_ID

        current_site_domain = request.META["HTTP_HOST"]
        try:
            current_site = Site.objects.get(domain__iexact=current_site_domain())
        except:
            current_site = Site.objects.get(id=site_id)
        set_current_site(current_site)

All of this works using the devserver, using the same settings as the wsgi server and configured with the same python path (as far as I can see)

At this point I'm hoping I've created an import loop somewhere (though that wouldn't make sense that it only occurs in the devserver)

edit: I found a list of the differences between devserver and apache in the django docs which says,

Devserver adds installed apps to sys.path, Apache does not.

Maybe that has something to do with it?

Answer 1

Have you set the CACHE_BACKEND in your settings.py? When DEBUG=True, this is not an issue, as I believe a dummy backend is installed, but in production, you will need to set this value, presumedley even if you're writing your own backend.

cache backend docs

If that's set and you're still having issues, try switching to the dummy cache module or the local memory one (even though this isn't a good production setup) to see if those work, and if they do, you may be missing a package in your WSGI's python path like python-memcached

Answer 2

问题的根源是sys.path我必须确保设置的mod_wsgi与开发服务器具有相同的sys.path，并且软件包的多余副本都留在了它们不属于的地方（由于重构不是t通过版本控制应用于服务器）