Django 1.7: tests and url patterns built from models raises “no such table”

Question

I'm trying to migrate a large Django project to version 1.7.

I've solved all the issues and the web site works smoothly, but I'm not able to run the tests suite anymore.

The problem arises from the app registry of Django, during the populate method. There are a few applications whose url patterns make use of the ORM at a rather early stage.

For example:

    url(r'^(?:(?P<platform_slug>%s)/)?(?:(?P<genre_slug>%s)/)?(?:(?P<year>%s)/)?$' % (
    r'|'.join([i['slug'] for i in Platform.active_objects.values("slug")]), 
    r'|'.join([i['slug'] for i in Genre.objects.values("slug")]),
    r'|'.join([str(i) for i in YEAR_FILTER_RANGE])),
    views.archive, name="gamecard_archive")

It's a complex pattern that allows for optional but distinguishable parameters, but it's not relevant for the question.

When I run the tests I got the infamous django.db.utils.OperationalError: no such table: contrib_platform because, apparently, the test runner still haven't create the database at that stage. The order in which the overall testing setup is executed is clearly different from 1.6.

While one solution could be to avoid accessing applications models in the url configuration during tests by replacing these rules with simpler versions checking sys.argv as in

if 'test' in sys.argv:
    [simple url pattern here]
else:
    [uber url pattern here]

I was wondering if anyone come up with a more elegant and portable solution.

UPDATE I thought to add the head of the traceback stack.
It shows that indeed it's not strictly related to tests, but to apps configuration.
I can't really say it's a regression, as Django 1.7 has been around for a while now and I doubt no one found it out before. Just a corner case of my project. Yet, in Django 1.6 it works like a charm, so clearly the database is accessible at an earlier stage.

Traceback (most recent call last):
  File "./manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "[...]/lib/python2.7/site-packages/django/core/management/__init__.py", line 385, in execute_from_command_line
utility.execute()
  File "[...]/lib/python2.7/site-packages/django/core/management/__init__.py", line 354, in execute
django.setup()
  File "[...]/lib/python2.7/site-packages/django/__init__.py", line 21, in setup
apps.populate(settings.INSTALLED_APPS)
  File "/Users/germanoguerrini/.virtualenvs/multi17/lib/python2.7/site-packages/django/apps/registry.py", line 115, in populate
app_config.ready()
  File "[...]/lib/python2.7/site-packages/django/contrib/admin/apps.py", line 22, in ready
self.module.autodiscover()
  File "[...]/lib/python2.7/site-packages/django/contrib/admin/__init__.py", line 23, in autodiscover
autodiscover_modules('admin', register_to=site)
  File "[...]/lib/python2.7/site-packages/django/utils/module_loading.py", line 74, in autodiscover_modules
import_module('%s.%s' % (app_config.name, module_to_search))
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/importlib/__init__.py", line 37, in import_module
__import__(name)
...

Answer 1

Generally I wouldn't consider building your urls from model instances a good practice. So probably try to find another solution for that, eg. by doing somehing like Genre.objects.filter(platform__slug=platform_slug) (don't know your exact model structure).

The problems you are experiencing are most likely due to the App-loading refactoring which was introduced with Django 1.7 and changes a lot in the way your models a loadeded etc. So you should try to put the url-changing code in the AppConfig.ready() method which is run after Django is finished with loading all necessary stuff. But nevertheless I would generally avoid to construct your urls based on data in the database like this.

Answer 2

Your solution (url configuration specified by a database table) works nice with Django 1.7 in a simple app. You have probably a complicated dependency or a cyclic import, that some your model require configured urls when it is imported. The mutual dependency can be dissolved eg by import inside the function when it is really used. Your solution with 'test' in sys.argv is also good, but better is to solve it without excluding tests, in order to you can write an integration test also your complicated code for urls.

It is strange that you have problems with test and not with other commands (eg python manage.py runserver) because your problem is very soon at setup, before there can be any difference between commands and before testrunner.

A standard solution is to define an attribute urls in your TesCase. For example a line urls = 'myapp.test_urls' . See docs test urlconf configuration . This can be useful later, after you fix your bug, or you should configure a fixture with test data for urls that seems overkill.