Django Rest 框架 - 减少查询并提高 SerializerMethodField() 的性能

Question

I have a model like this with relationship Booking -- Payment (one to may)

(one Booking can have many Payments)

My problem is that I calling too many expensive queries because of SerializerFieldModel().

class Booking(AbstractItem):
    accommodation = models.CharField(max_length=100)
    room = models.CharField(max_length=100)
    booking_method = models.CharField(max_length=100)

class Payment(AbstractItem):
    booking = models.ForeignKey(Booking, on_delete=models.CASCADE)
    # paid, refund
    status = models.PositiveIntegerField(choices=core_choices.PAYMENT_STATUS, default=core_choices.PAID, null=True)
    # cash, credit_card, transfer
    payment_method = models.PositiveIntegerField(choices=core_choices.PAYMENT_METHOD, default=core_choices.CASH)
    price = models.DecimalField(max_digits=10, decimal_places=0, null=True)

This is my serializer

class BookingXLSXSerializer(ModelSerializer):
    paid_cash = SerializerMethodField()
    paid_card = SerializerMethodField()
    refund_cash = SerializerMethodField()
    refund_card = SerializerMethodField()

    class Meta:
        model = Booking
        fields = ('id', 'accommodation ', 'room', 'booking_method', 'paid_cash', 'paid_card', 'refund_cash', 'refund_card')


    def get_paid_cash(self, obj):
        payments = Payment.objects.filter(booking=obj.id, status=core_choices.CASH)
        cash = 0
        for payment in payments:
           cash += payment.price
        return cash

    #I noticed that many of the def could use the .all() query then just filter it out
...

this is my view:

class MyExampleViewSet(ListAPIView):
    queryset = Booking.objects.all()
    serializer_class = BookingXLSXSerializer
    pagination_class = SuperLargePagination

I noticed that many SerializerMethodField() could share query and use different filter. Is there a smarter way to reduce calling queries for SerializerMethodField(). Or maybe a way to share the query?

Answer 1

prefetch_related()

You can try adding a prefetch_related clause:

queryset = Booking.objects.prefetch_related('payment_set').all()

To benefit from this, you need to change the code in your serializer to actually use the related field, for example:

cash = obj.payment_set.filter(status=core_choices.CASH).aggregate(Sum('price'))['price_sum']

This will still result in two queries, though. annotate is more precise.

See also Django: Does prefetch_related() follow reverse relationship lookup?

annotate()

Another, more complex possibility which also gives you more influence what the DB is actually returning to you, are annotations :

This will reduce it to one query letting the DB do all of the work. I'm writing this down without testing, you will have to check the details out for yourself.

It is definitly not easy to create complex annotations but it is a great tool for improving performance, and you can find a lot of good code for it already on this site. (And it can actually be fun...)

from django.db.models import OuterRef, Subquery, IntegerField

class MyExampleViewSet(ListAPIView):
    cash_sub_qs = Payment.objects.filter(
        booking=OuterRef('pk'),
        status=core_choices.CASH
    ).annotate(paid_cash=Sum('price')).values('paid_cash')[:1]
    queryset = Booking.objects.annotate(
        paid_cash=Subquery(cash_sub_qs, output_field=DecimalField()),
        refund_cash=..., # missing from your question, prolly also a sum?
    ).all()
    serializer_class = BookingXLSXSerializer
    pagination_class = SuperLargePagination

Your serializer should only use the queryset call as a fallback:

class BookingXLSXSerializer(ModelSerializer):
    paid_cash = SerializerMethodField()
    paid_card = SerializerMethodField()
    refund_cash = SerializerMethodField()
    refund_card = SerializerMethodField()

    class Meta:
        model = Booking
        fields = ('id', 'accommodation ', 'room', 'booking_method', 'paid_cash', 'paid_card', 'refund_cash', 'refund_card')


    def get_paid_cash(self, obj):
        cash = getattr(obj, 'paid_cash', None) # this is the annotated field
        if cash is None:
            cash = obj.payment_set.filter(status=core_choices.CASH).aggregate(Sum('price'))['price_sum']
        return cash