Django Many to Many and admin

Question

I have a django ap which has a rather complicated model setup. I ended up using multi level composition to create a hierarchical model. All the relations are one to one, so I could have use inheritance but I chose not to so that i would benefit from having object composition for my models, this means I can do things like

product.outerframe.top.cost

which make the complicated calculations I have to preform, a lot better organised.

However, This model arrangement makes using the django admin tricky. I basically have a through table, ie the outerframe table is just a bunch of foreign keys to other tables (with unique constraint on each). I ended up oerriding the add_view() and change_view() methods of ModelAdmin, which is pretty hard.

Is there an easier way to deal with many to many / through tables when using the django admin?

The tables are arranged like so:

Product > outerframe, innerframe, glass, other

outerframe > top, bottom, side etc.

innerframe > top, bottom, side etc.

glass > glass_type etc.

other > accessories etc.

Here are my models:

class Product(mixins.ProductVariables):
    name = models.CharField(max_length=255)
    sku = models.CharField(max_length=100, unique=True, db_index=True)
    image = thumbnail.ImageField(upload_to='product_images', blank=True)
    description = models.TextField(blank=True)
    group = models.ForeignKey('ProductGroup', related_name='products', null=True)
    hidden = models.BooleanField(default=False)
    product_specific_mark_up = models.DecimalField(default=1.0, max_digits=5,decimal_places=2)

    # Methods for totals
    def total_material_cost(self, width, height, options):
        return sum([
            self.outerframe.cost(width, height, options),
            self.innerframe.cost(width, height, options),
            self.glass.cost(width, height, options),
            self.other.cost(width, height, options),
        ])

    def total_labour_time(self, width, height, options):
        return sum([
            self.outerframe.labour_time(width, height, options),
            self.innerframe.labour_time(width, height, options),
            self.glass.labour_time(width, height, options),
            self.other.labour_time(width, height, options),
        ])

    def total_co2_output(self, width, height, options):
        return sum([
            self.outerframe.co2_output(width, height, options),
            self.innerframe.co2_output(width, height, options),
            self.glass.co2_output(width, height, options),
            self.other.co2_output(width, height, options),
        ])

    @property
    def max_overall_width(self):
        return 1000

    @property
    def max_overall_height(self):
        return 1000

    def __unicode__(self):
        return self.name


class OuterFrame(models.Model, mixins.GetFieldsMixin, mixins.GetRelatedClassesMixin):
    top = models.OneToOneField(mixins.TopFrame)
    bottom = models.OneToOneField(mixins.BottomFrame)
    side = models.OneToOneField(mixins.SideFrame)
    accessories = models.OneToOneField(mixins.Accessories)
    flashing = models.OneToOneField(mixins.Flashing)
    silicone = models.OneToOneField(mixins.Silicone)

    product = models.OneToOneField(Product)

    def cost(self, width, height, options):
        #accessories_cost = (self.accessories.cost if options['accessories'] else 0)
        #flashing_cost = (self.flashing.cost if options['flashing'] else 0)
        #silicone_cost = (self.silicone.cost if options['silicone'] else 0)
        return sum([
            self.top.cost * (width / 1000),
            self.bottom.cost * (width / 1000),
            self.side.cost * (width*2 / 1000),
            #accessories_cost,
            #flashing_cost,
            #silicone_cost,
        ])

    def labour_time(self, width, height, options):
        return datetime.timedelta(minutes=100)

    def CO2_output(self, width, height, options):
        return 100 # some kg measurement

    @classmethod
    def get_fields(cls):
        options = cls._meta
        fields = {}
        for field in options.fields:
            if field.name == 'product':
                continue
            if isinstance(field, models.OneToOneField):
                related_cls = field.rel.to
                related_fields = fields_for_model(related_cls, fields=related_cls.get_fields())
                fields.update( { related_cls.__name__ + '_' + name:field for name, field in related_fields.iteritems() })
        return fields



class InnerFrame(models.Model, mixins.GetFieldsMixin, mixins.GetRelatedClassesMixin):
    top = models.OneToOneField(mixins.TopFrame)
    bottom = models.OneToOneField(mixins.BottomFrame)
    side = models.OneToOneField(mixins.SideFrame)
    accessories = models.OneToOneField(mixins.Accessories)

    product = models.OneToOneField(Product)

    def cost(self, width, height, options):
        #accessories_cost = (self.accessories.cost if options['accessories'] else 0)
        print self.top.cost
        return sum([
            self.top.cost * (width / 1000),
            self.bottom.cost * (width / 1000),
            self.side.cost * (width*2 / 1000),
        #    accessories_cost,
        ])

    def labour_time(self, width, height, options):
        return datetime.timedelta(minutes=100)

    def CO2_output(self, width, height, options):
        return 100 # some kg measurement

class Glass(models.Model, mixins.GetRelatedClassesMixin):
    glass_type_a = models.OneToOneField(mixins.GlassTypeA)
    glass_type_b = models.OneToOneField(mixins.GlassTypeB)
    enhanced = models.OneToOneField(mixins.Enhanced)
    laminate = models.OneToOneField(mixins.Laminate)
    low_iron = models.OneToOneField(mixins.LowIron)
    privacy = models.OneToOneField(mixins.Privacy)
    anti_slip = models.OneToOneField(mixins.AntiSlip)
    heat_film_mirror = models.OneToOneField(mixins.HeatMirrorField)
    posished_edges = models.OneToOneField(mixins.PolishedEdges)

    product = models.OneToOneField(Product)

    def cost(self, width, height, options):
        return sum([
        ])

    def labour_time(self, width, height, options):
        return datetime.timedelta(minutes=100)

    def CO2_output(self, width, height, options):
        return 100 # some kg measurement

class Other(models.Model, mixins.GetRelatedClassesMixin):
    num_packages = models.OneToOneField(mixins.NumberPackages)

    product = models.OneToOneField(Product)

    def cost(self, width, height, options):
        return 100

    def labour_time(self, width, height, options):
        return datetime.timedelta(minutes=100)

    def CO2_output(self, width, height, options):
        return 100 # some kg measurement

mixins:

class TimeCostMixin(models.Model, GetFieldsMixin):
    cost = models.DecimalField(default=0.0, max_digits=10, decimal_places=2)
    time = models.TimeField(default=datetime.timedelta(0))
    class Meta:
        abstract = True

##### Frame #####
class FrameComponentMixin(TimeCostMixin):
    external_width = models.IntegerField(default=0)
    material_weight = models.DecimalField(default=0.0, max_digits=10,decimal_places=2)
    u_value = models.DecimalField(default=0.0, max_digits=10,decimal_places=2)
    class Meta:
        abstract = True


class TopFrame(FrameComponentMixin):
    pass


class BottomFrame(FrameComponentMixin):
    pass


class SideFrame(FrameComponentMixin):
    pass


class Accessories(TimeCostMixin):
    material_weight = models.DecimalField(default=0.0,max_digits=10,decimal_places=2)


class Flashing(TimeCostMixin):
    pass


class Silicone(TimeCostMixin):
    labour_time = models.DecimalField(default=0.0, max_digits=10,decimal_places=2)
#################

##### Glass #####
class GlassTypeA(TimeCostMixin):
    material_weight = models.DecimalField(default=0.0, max_digits=10,decimal_places=2)
    u_value = models.DecimalField(default=0.0, max_digits=10,decimal_places=2)

class GlassTypeB(TimeCostMixin):
    material_weight = models.DecimalField(default=0.0, max_digits=10,decimal_places=2)
    u_value = models.DecimalField(default=0.0, max_digits=10,decimal_places=2)

class Enhanced(TimeCostMixin):
    material_weight = models.DecimalField(default=0.0, max_digits=10,decimal_places=2)

class Laminate(TimeCostMixin):
    material_weight = models.DecimalField(default=0.0, max_digits=10,decimal_places=2)

class LowIron(TimeCostMixin):
    pass

class Privacy(TimeCostMixin):
    pass

class AntiSlip(TimeCostMixin):
    pass

class HeatMirrorField(TimeCostMixin):
    u_value = models.DecimalField(default=0.0, max_digits=10,decimal_places=2)

class PolishedEdges(models.Model):
    cost = models.DecimalField(default=0.0, max_digits=10, decimal_places=2)
##################

##### other  #####
class NumberPackages(models.Model):
    number_of_packages = models.IntegerField(default=0)
##################

and a hair pulling admin!

class ProductAdmin(AdminImageMixin, admin.ModelAdmin):
    inlines = [ProductDownloadInline, ProductConfigurationInline]

    add_form_template = 'admin/products/add_form.html'
    change_form_template = 'admin/products/add_form.html'


    @csrf_protect_m
    @transaction.atomic
    def add_view(self, request, form_url='', extra_context=None):
        extra_context = extra_context or {}

        "The 'add' admin view for this model."
        model = self.model
        opts = model._meta

        if not self.has_add_permission(request):
            raise PermissionDenied

        ModelForm = self.get_form(request)
        formsets = []
        inline_instances = self.get_inline_instances(request, None)
        if request.method == 'POST':
            form = ModelForm(request.POST, request.FILES)
            if form.is_valid():
                new_object = self.save_form(request, form, change=False)
                form_validated = True
            else:
                form_validated = False
                new_object = self.model()
            prefixes = {}
            for FormSet, inline in zip(self.get_formsets(request), inline_instances):
                prefix = FormSet.get_default_prefix()
                prefixes[prefix] = prefixes.get(prefix, 0) + 1
                if prefixes[prefix] != 1 or not prefix:
                    prefix = "%s-%s" % (prefix, prefixes[prefix])
                formset = FormSet(data=request.POST, files=request.FILES,
                                  instance=new_object,
                                  save_as_new="_saveasnew" in request.POST,
                                  prefix=prefix, queryset=inline.get_queryset(request))
                formsets.append(formset)

            #####
            outer_frame_forms = [
                modelform_factory(cls)(request.POST, prefix='OuterFrame_'+cls.__name__)
                for cls in models.OuterFrame.get_related_classes(exclude=['product'])
            ]
            inner_frame_forms = [
                modelform_factory(cls)(request.POST, prefix='InnerFrame'+cls.__name__)
                for cls in models.InnerFrame.get_related_classes(exclude=['product'])
            ]
            glass_forms = [
                modelform_factory(cls)(request.POST, prefix='InnerFrame'+cls.__name__)
                for cls in models.Glass.get_related_classes(exclude=['product'])
            ]
            other_forms = [
                modelform_factory(cls)(request.POST, prefix='InnerFrame'+cls.__name__)
                for cls in models.Other.get_related_classes(exclude=['product'])
            ]
            #####

            if all_valid(formsets
                        +outer_frame_forms
                        +inner_frame_forms
                        +glass_forms
                        +other_forms
                        ) and form_validated:
                self.save_model(request, new_object, form, False)
                self.save_related(request, form, formsets, False)
                self.log_addition(request, new_object)

                ##### save object hierichy #####
                # inner frame
                inner_frame = models.InnerFrame()
                inner_frame.product = new_object
                mapping = {f.rel.to:f.name for f in models.InnerFrame._meta.fields if f.name not in ['id','product']}
                for f in inner_frame_forms:
                    obj = f.save()
                    setattr(inner_frame, mapping[obj.__class__], obj)
                inner_frame.save()
                # outer frame
                outer_frame = models.OuterFrame()
                outer_frame.product = new_object
                mapping = {f.rel.to:f.name for f in models.OuterFrame._meta.fields if f.name not in ['id','product']}
                for f in outer_frame_forms:
                    obj = f.save()
                    setattr(outer_frame, mapping[obj.__class__], obj)
                outer_frame.save()
                # glass
                glass = models.Glass()
                glass.product = new_object
                mapping = {f.rel.to:f.name for f in models.Glass._meta.fields if f.name not in ['id','product']}
                for f in glass_forms:
                    obj = f.save()
                    setattr(glass, mapping[obj.__class__], obj)
                glass.save()
                # other
                other = models.Other()
                other.product = new_object
                mapping = {f.rel.to:f.name for f in models.Other._meta.fields if f.name not in ['id','product']}
                for f in other_forms:
                    obj = f.save()
                    setattr(other, mapping[obj.__class__], obj)
                other.save()
                #################################

                return self.response_add(request, new_object)
        else:
            forms = SortedDict({})
            forms['Outer Frame Variables'] = {
                cls.__name__: modelform_factory(cls)(prefix='OuterFrame_'+cls.__name__)
                for cls in models.OuterFrame.get_related_classes(exclude=['product'])
            }
            forms['Inner Frame Variables'] = {
                cls.__name__: modelform_factory(cls)(prefix='InnerFrame'+cls.__name__)
                for cls in models.InnerFrame.get_related_classes(exclude=['product'])
            }
            forms['Glass Variables'] = {
                cls.__name__: modelform_factory(cls)(prefix='InnerFrame'+cls.__name__)
                for cls in models.Glass.get_related_classes(exclude=['product'])
            }
            forms['Other Variables'] = {
                cls.__name__: modelform_factory(cls)(prefix='InnerFrame'+cls.__name__)
                for cls in models.Other.get_related_classes(exclude=['product'])
            }
            extra_context['forms'] = forms

            # Prepare the dict of initial data from the request.
            # We have to special-case M2Ms as a list of comma-separated PKs.
            initial = dict(request.GET.items())
            for k in initial:
                try:
                    f = opts.get_field(k)
                except models.FieldDoesNotExist:
                    continue
                if isinstance(f, models.ManyToManyField):
                    initial[k] = initial[k].split(",")
            form = ModelForm(initial=initial)
            prefixes = {}
            for FormSet, inline in zip(self.get_formsets(request), inline_instances):
                prefix = FormSet.get_default_prefix()
                prefixes[prefix] = prefixes.get(prefix, 0) + 1
                if prefixes[prefix] != 1 or not prefix:
                    prefix = "%s-%s" % (prefix, prefixes[prefix])
                formset = FormSet(instance=self.model(), prefix=prefix,
                                  queryset=inline.get_queryset(request))
                formsets.append(formset)

        adminForm = helpers.AdminForm(form, list(self.get_fieldsets(request)),
            self.get_prepopulated_fields(request),
            self.get_readonly_fields(request),
            model_admin=self)
        media = self.media + adminForm.media

        inline_admin_formsets = []
        for inline, formset in zip(inline_instances, formsets):
            fieldsets = list(inline.get_fieldsets(request))
            readonly = list(inline.get_readonly_fields(request))
            prepopulated = dict(inline.get_prepopulated_fields(request))
            inline_admin_formset = helpers.InlineAdminFormSet(inline, formset,
                fieldsets, prepopulated, readonly, model_admin=self)
            inline_admin_formsets.append(inline_admin_formset)
            media = media + inline_admin_formset.media

        context = {
            'title': _('Add %s') % force_text(opts.verbose_name),
            'adminform': adminForm,
            'is_popup': IS_POPUP_VAR in request.REQUEST,
            'media': media,
            'inline_admin_formsets': inline_admin_formsets,
            'errors': helpers.AdminErrorList(form, formsets),
            'app_label': opts.app_label,
            'preserved_filters': self.get_preserved_filters(request),
        }
        context.update(extra_context or {})
        return self.render_change_form(request, context, form_url=form_url, add=True)

Answer 1

I haven't fully processed your lengthy add_view method, but the answer to your general question is simply "No." The admin doesn't provide any good way to handle multi-layer heterogeneous hierarchies. Two-layer hierarchies are handled nicely by inlines, and so you can easily make it so that from editing an object in any one layer, you can conveniently manage related objects in the next layer down; but nothing beyond that.

There has been a ticket open for years to add nested-inline support to the admin, which would help to handle this situation. But there are lots of tricky edge-cases and it's very hard to make the UI understandable, so the patch has never reached a commit-ready state.

At some point the complexity of your data model is just beyond what the generic admin interface can handle with good usability, and you're better off just writing your own customized admin interface. Mostly the admin is just built on top of ModelForms and InlineModelFormsets, so it's not as hard as you might think to just build your own that works the way you want; it's often easier (and with better results) than trying to heavily customize the admin.

I should also mention that it is possible to use admin inlines for many-to-many through tables (even if the through table is implicit, not its own model class), as it's not immediately obvious how to access the implicitly-created through model:

class MyM2MInline(admin.TabularInline): model = SomeModel.m2m_field.through