django-rest-framework (bulk): How to implement nonstandard bulk load update/create

Question

I have been trying to find implement a way to handle some uploaded data in DRF (with or without Bulk) in order to keep it consistent with my other APIs and return nice results, but it seems like it might be more trouble than it's worth.

The system handles referential data from external systems, which should not need to know anything about how data is stored or what already exists there. As such the JSON data to upload looks like:

    data = [
        {"name": "foo", 'key': 'FOO', "resource_type": 'some-resource', 'team_tag': 'ABC1', 'properties': {"foo": "bar"}},
        {"name": "foo2", 'key': 'FOO2', "resource_type": 'some-resource', 'team_tag': 'ABC2'}
    ]

As you can see there are no IDs - key and resource_type are a unique tuple (essentially a natural key). The system should either update or create based on whether these already exist. However I am running into problems due to assumptions about:

• all resources being either updated OR created (due to POST/PUT/PATCH)
• that the data will have an 'id' attribute, and there doesn't seem to be nice entry points to change how objects are queried/keyed
• the way that serializers are created, validated, and saved, especially as to whether they are to be updated or created.

I made this work earlier by having my ViewSet delete ALL the relevant objects from the database, then DRF could simply insert new ones. That was a quick hack which is inefficient and does not allow for maintaining any changes in the system.

In trying to implement this, it seems I am overriding a LOT of methods in ViewSets and Serializers just to make it compatible, much more than I would need at minimum without DRF, but I would like to keep it consistent if possible.

I also ran into the issue at https://github.com/miki725/django-rest-framework-bulk/issues/30 and implemented their suggestions, however ran into other problems, mainly with their use of both BulkListSerializer and a custom model serializer, which would both need to be overridden.

Finally, I will need to add something to the view to delete or mark any objects which are now orphaned (do not exist externally).

Is there a known pattern for this, can this be worked into DRF(B), or should I just write my own independent view?

Thanks!

Answer 1

It looks like you are trying to use non model data with ModelSerializer .

You may want to perform the [de]serialization with a regular serializer instead. See my post about that topic .