Annotating query with sum of multiplication of two fields from a related model in Django

Question

Relevant models are:

class Score(models.Model):
    chouette = models.ForeignKey(
        Chouette, on_delete=models.CASCADE, related_name="scores"
    )
    game = models.IntegerField(blank=True, null=True)
    player = models.ForeignKey(User, on_delete=models.CASCADE, related_name="scores")
    position = models.CharField(max_length=10, blank=True, null=True)
    score = models.DecimalField(max_digits=10, decimal_places=3, blank=True, null=True)

    objects = ScoreManager()

class User(AbstractUser):
    name = CharField(_("Name of User"), blank=True, max_length=255)
    handle = CharField(max_length=10)
    xp = IntegerField(default=0)
    objects = CustomUserManager()

I have a custom manager for User that builds a queryset with a lot of annotations, but the one I am struggling with relates to Score. I am trying to calculate a total_winnings annotation that does a subquery on each User and multiplies the score field in Score for each Score instance associated with a user by a field in the Chouette model (not shown--just a decimal field called stake). I have tried several variations of this as my subquery:

def build_chwinnings(self):
    total_winnings = self.filter(pk=OuterRef("pk")).annotate(
        total_winnings = Sum(F("scores__score") * F("scores__chouette__stake")
        )
    )
    return self.annotate(total_winnings=Subquery(total_winnings.values("total_winnings"))

But they all result in the following exception:

django.db.utils.ProgrammingError: more than one row returned by a subquery used as an expression

The "easiest" way to do this would be to add the calculated value (scores__score * scores__chouette__stake) as a field to the Score model, but all the boo birds hate duplicating data within the database. Adding it as a property to Score or to the queryset for Score don't work, because those aren't accessible through the subquery.

Is there a pretty way to do this, or does it make the most sense to add a field in Score and update it on every save? (The stake value in Chouette doesn't typically get changed after the fact, so I'm unlikely to have to worry about any signals, and in reality, the Score instances are similarly "fixed" after their creation.)

Answer 1

You are probably missing the group by on the player column in the subquery. Here is an example that should work. You can adapt this to work in your Manager.

from django.db.models import Subquery, Sum, F, OuterRef

tw_subquery = Subquery(Score.objects.values(
    'player'  # Required to group the annotation by the player
).annotate(
    total_winnings = Sum(F("score") * F("chouette__stake")
).filter(
    player_id=OuterRef('id')
).values(
    'total_winnings'  # required to select only one column
))

users = User.objects.annotate(total_winnings=tw_subquery)

This kind of subquery is handled for you by the django-sql-utils package . This will allow you to do:

from sql_util import SubquerySum

tw_subquery = SubquerySum(F("scores__score") * F("scores__chouette__stake"))
users = User.objects.annotate(total_winnings=tw_subquery)

As a totally separate issue, it looks to me like in your build_chwinnings method that

total_winnings = self.filter(...)

is not right. This queryset should be on the Score model, but self is going to be a queryset on the User model if this is a Manager for the User model. But I'm not sure I'm seeing enough of the code to be certain of this.