How to annotate Django QuerySet with other object using Subquery

Question

In Django version 1.11, Subquery expressions were added. I was hoping to use this feature to select a related model object based on some filters.

This is an example from the documentation :

from django.db.models import OuterRef, Subquery
newest = Comment.objects.filter(post=OuterRef('pk')).order_by('-created_at')   
Post.objects.annotate(newest_commenter_email=Subquery(newest.values('email')[:1]))

I would like to do the same, but instead of just annotating with the " newest_commenter_email " in this scenario, I'd like the whole Comment object saved to a newest_comment annotation, like so:

from django.db.models import OuterRef, Subquery
newest = Comment.objects.filter(post=OuterRef('pk')).order_by('-created_at')   
Post.objects.annotate(newest_comment=Subquery(newest[:1]))

However, this results in the following error:

FieldError: Expression contains mixed types. You must set output_field

Is there a way around this?

Answer 1

You ommited values on the subquery, try this:

Post.objects.annotate(newest_commenter=Subquery(newest.values('author')[:1]))

where 'author' is the "commenter" field on Comment model.

Answer 2

You can't do it. You can only annotate with a basic type values, not with the object. The basic reason for this is that result of SQL query is a table with all its cells are basic values, not nested tables. In order to do what you want we need SQL to support nested tables and Django to be able to convert nested tables into nested objects. I don't think that's ever going to happen. So, in order to get several fields of related object you should repeat annotation for each field:

Post.objects.annotate(
    newest_commenter_email=Subquery(newest.values('email')[:1]),
    newest_commenter_username=Subquery(newest.values('username')[:1]),
)

Don't be afraid of performance issues due to executing the same query for each field. SQL engine is smart enough to see that query is the same and to cache its value

Answer 3

As suggested by Brobin -

Comment.objects.filter(post=pk).order_by('created_at').select_related('post')[:1]

If you really want to go from the Post model though do this -

q = (
  Comment
  .objects
  .filter(
    pk=Subquery(
      Comment
     .objects
     .filter(post=OuterRef('post'))
     .order_by('-created')
     .values('pk')
     [:1]
    )
  )
)

Post.objects.prefetch_related(Prefetch('comment_set', queryset=q)).all()

Unfortunately we cannot slice in the outer query for q because django disallows it. So we slice inside a subquery and then filter on the outer query.

Answer 4

You want to filter of the related model, hence this case I feed Prefetch() is one of the best options I can think about

You can read about it from here Django1.1 Prefetch Objects

and this is an example of what you're looking for

from django.db.models import Prefetch
newest = Comment.objects.filter(do_your_filters_here)   
Post.objects.prefetch_related(Prefetch("comments", queryset=newest, to_attr="newest_comments"))