Running a standalone script doing a model query in Django with `settings/dev.py` instead of `settings.py`

Question

Note the settings/dev.py instead of one settings.py file and the script.py in my_app in the following Django(1.4.3) project:

.
├── my_project
│   ├── my_app
│   │   ├── __init__.py
│   │   ├── models.py
│   │   ├── tests.py
│   │   ├── views.py
│   │   └── script.py
│   ├── __init__.py
│   ├── settings
│   │   ├── dev.py
│   │   ├── __init__.py
│   │   └── prod.py
│   ├── urls.py
│   └── wsgi.py
├── manage.py
└── requirements.txt

When I only had one settings.py file in place of the settings folder I was able to run the following script without any errors :

script.py:

###################################################################
# set up for making it possible to run a model query from my script. 
###################################################################
import os
import sys
sys.path.append(os.path.abspath(os.path.join(os.path.dirname(__file__), os.path.pardir)))
from django.core.management import setup_environ
import settings
setup_environ(settings)
####################################################################  
from my_app.models import MyModel

all_entries = MyModel.objects.all()

Btw, I got that from the second approach in this article , my setup is a few lines longer because my script.py is in the my_app folder instead of directly under the my_project folder.

Because I am now using settings/dev.py instead of settings.py I changed the last two lines of my setup in my script to the following:

import settings.dev
import setup_environ(settings.dev)

But when I run my script now I get this error:

Traceback (most recent call last):
  File "my_script.py", line 12, in <module>
    all_entries = MyModel.objects.all()
  File "/home/my_username/.virtualenvs/my_project/local/lib/python2.7/site-packages/django/db/models/manager.py", line 131, in get
    return self.get_query_set().get(*args, **kwargs)
  File "/home/my_username/.virtualenvs/my_project/local/lib/python2.7/site-packages/django/db/models/query.py", line 358, in get
    clone = self.filter(*args, **kwargs)
  File "/home/my_username/.virtualenvs/my_project/local/lib/python2.7/site-packages/django/db/models/query.py", line 624, in filter
    return self._filter_or_exclude(False, *args, **kwargs)
  File "/home/my_username/.virtualenvs/my_project/local/lib/python2.7/site-packages/django/db/models/query.py", line 642, in _filter_or_exclude
    clone.query.add_q(Q(*args, **kwargs))
  File "/home/my_username/.virtualenvs/my_project/local/lib/python2.7/site-packages/django/db/models/sql/query.py", line 1250, in add_q
    can_reuse=used_aliases, force_having=force_having)
  File "/home/my_username/.virtualenvs/my_project/local/lib/python2.7/site-packages/django/db/models/sql/query.py", line 1122, in add_filter
    process_extras=process_extras)
  File "/home/my_username/.virtualenvs/my_project/local/lib/python2.7/site-packages/django/db/models/sql/query.py", line 1304, in setup_joins
    field, model, direct, m2m = opts.get_field_by_name(name)
  File "/home/my_username/.virtualenvs/my_project/local/lib/python2.7/site-packages/django/db/models/options.py", line 311, in get_field_by_name
    cache = self.init_name_map()
  File "/home/my_username/.virtualenvs/my_project/local/lib/python2.7/site-packages/django/db/models/options.py", line 341, in init_name_map
    for f, model in self.get_all_related_m2m_objects_with_model():
  File "/home/my_username/.virtualenvs/my_project/local/lib/python2.7/site-packages/django/db/models/options.py", line 429, in get_all_related_m2m_objects_with_model
    cache = self._fill_related_many_to_many_cache()
  File "/home/my_username/.virtualenvs/my_project/local/lib/python2.7/site-packages/django/db/models/options.py", line 443, in _fill_related_many_to_many_cache
    for klass in get_models(only_installed=False):
  File "/home/my_username/.virtualenvs/my_project/local/lib/python2.7/site-packages/django/db/models/loading.py", line 181, in get_models
    self._populate()
  File "/home/my_username/.virtualenvs/my_project/local/lib/python2.7/site-packages/django/db/models/loading.py", line 64, in _populate
    self.load_app(app_name, True)
  File "/home/my_username/.virtualenvs/my_project/local/lib/python2.7/site-packages/django/db/models/loading.py", line 86, in load_app
    app_module = import_module(app_name)
  File "/home/my_username/.virtualenvs/my_project/local/lib/python2.7/site-packages/django/utils/importlib.py", line 35, in import_module
    __import__(name)
ImportError: No module named my_project.my_app

Why is this error occurring? How can I run my script in Django with settings/dev.py instead of settings.py ?

Answer 1

If you're looking to just run a script in the django environment, then the simplest way to accomplish this is to create a ./manage.py subcommand, like this

from django.core.management.base import BaseCommand
from my_app.models import MyModel

class Command(BaseCommand):
    help = 'runs your code in the django environment'

    def handle(self, *args, **options):
        all_entries = MyModel.objects.all()
        for entry in all_entries:
            self.stdout.write('entry "%s"' % entry)

The docs are quite helpful with explaining this.

However, you can specify a settings file to run with using

$ django-admin.py runserver --settings=settings.dev

which will run the test server using the settings in dev however, I fear your problems are more deep seated than simply that. I wouldn't recommend ever changing the manage.py file as this can lead to inconsistencies and future headaches.

Note also that dev.py should be a complete settings file if you are to do this. I would personally recommend a structure like this:

|-settings
|    |- __init__.py
|    |- base.py
|    |- dev.py
|    |- prod.py

and keep all the general settings in your base.py and change the first line of your dev.py etc to something like

# settings/dev.py
from .base import *

DEBUG = True
...

EDIT

If you're just looking to test things out, why not try

$ ./manage.py shell

or with your dev settings

$ django-admin.py shell --settings=settings.dev

as this will set all the OS environment variables, settings.py for you, and then you can test / debug with

>>> from my_app.models import MyModel
>>> all_entries = MyModel.objects.all()
>>> for entry in all_entries:
...   print entry