django fixtures（来自dumpdata）在测试时失败

Question

I'm trying to dump data from my production server to use as test on my dev server, but I'm getting errors when running "./manage.py test" on the dev server specifing the fixture file created on the prod server.

Here are the tries I made based on google/stackoverflow search:

# python manage.py dumpdata --indent=4 --natural
error when running tests: IntegrityError: (1062, "Duplicate entry 'cms-agencies' for key 'app_label'")

# python manage.py dumpdata --exclude contenttypes --indent=4
error when running tests: IntegrityError: (1452, 'Cannot add or update a child row: a foreign key constraint fails (`test_current`.`django_admin_log`, CONSTRAINT `content_type_id_refs_id_288599e6` FOREIGN KEY (`content_type_id`) REFERENCES `django_content_type` (`id`))')

# python manage.py dumpdata --exclude contenttypes --natural --indent=4
error when running tests: IntegrityError: (1062, "Duplicate entry '14-add_agencies' for key 'content_type_id'")

# python manage.py dumpdata --exclude contenttypes --exclude auth --indent=4
error when running tests: IntegrityError: (1452, 'Cannot add or update a child row: a foreign key constraint fails (`test_current`.`django_admin_log`, CONSTRAINT `user_id_refs_id_c8665aa` FOREIGN KEY (`user_id`) REFERENCES `auth_user` (`id`))')

# python manage.py dumpdata --exclude contenttypes --exclude auth --natural --indent=4
error when running tests: IntegrityError: (1452, 'Cannot add or update a child row: a foreign key constraint fails (`test_current_abril`.`django_admin_log`, CONSTRAINT `user_id_refs_id_c8665aa` FOREIGN KEY (`user_id`) REFERENCES `auth_user` (`id`))')

I also tried removing "'init_command': 'SET storage_engine=INNODB'" from settings.py and still got 1062 errors.

I don't understand this problem. Shouldn't django just recreate the DB exactly as it was on the prod server when I load the fixtures?

Answer 1

I had similar problems and these args worked for me:

python manage.py dumpdata --natural --exclude auth.permission --exclude contenttypes --indent 4

I also had a lot of issues with the post_save signal creating objects twice. There's a fix for that: How do I prevent fixtures from conflicting with django post_save signal code?

Answer 2

I believe the errors are telling you exactly what is happening. Is app_label unique? I would guess it is. I would think you have two objects with the same app_label key value. ( cms-agencies )

Additionally when you have a foreign key relationship you need to have the object that corresponds to the foreign key. Dump data doesn't do this because it only dumps one model.

Something like https://github.com/davedash/django-fixture-magic is great for this. It dumps your model AND all fk dependencies.

Answer 3

The problem is, that if you use natural keys (natural-foreign keys in later versions of Django), Django will actually store many-to-many relationships inside parent objects. That is something you want. But then you cannot simply dump all tables, you must not include the many-to-many tables/models in your dump as you would then load same data twice - and boom, duplicates and IntegrityErrors.

For example, you should dump auth.User and auth.Group , but not auth.User_Groups . Look at example of a dump from Django 1.7:

{
    "model": "auth.group",
    "fields": {
        "permissions": [],
        "name": "ST"
    },
},
{
    "model": "auth.group",
    "fields": {
        "permissions": [],
        "name": "property_manager"
    },
},

{
    "model": "auth.user",
    "fields": {
        "username": "boss",
        "groups": [
            ["property_manager"],["ST"],
        ],
        "user_permissions": [],
    },
},

Following line creates a comprehensive dump of users/permissions and content types you can move to dev to get identical copy, including identical row order and primary keys (tested on Django 1.7):

python manage.py dumpdata auth.User auth.Group contenttypes auth.Permission --indent 4 --natural-foreign > users.json

Answer 4

your problem might be that you dumped out a unicode file but django expects a ascii file format when loading. This will happen using PowerShell on Windows if that applies to you.