Django annotate related field

Question

I want to optimize a simple django query to prefetch all the latest values.

sensors = Sensor.objects.all()

Here is the model:

class Sensor:
    last_record_at = models.DateTimeField(null=True, blank=True)

class Record:
    value = models.FloatField()
    sensor = models.ForeignKey('hardware.Sensor', on_delete=models.CASCADE)
    created = models.DateTimeField(auto_now_add=True)

Before, Sensor model had a foreign key (last record) to record and all the records could be retrieved simply by adding:

.select_related('last_record')

To optimize the database, I removed that foreign key and replaced it with a datetime field named last_record_at.

I am using Django 2.0 and I am wondering if there is a pretty ORM way to retrieve all the sensors and last records in one (or two) query using subqueries, annotation or prefetching.

Something like (it does not work):

record_sub_query = Record.objects.filter(
    created=OuterRef('last_record_at'), 
    sensor_id=OuterRef('pk')
) 
sensors = Sensor.objects.all().annotate(
    last_record=Subquery(record_sub_query[:1])
)   

or using prefetch:

sensors = Sensor.objects.all().prefetch_related(
    Prefetch(
        'records', 
        queryset=Record.objects.filter(created=F('sensor__last_record_at'))
    )
)

The only alternative I see is writing Raw SQL for this rather simple problem.

Answer 1

I would do it the other way around: you currently try to retrieve all Sensors , then get their latest associated Record . Why not retrieve all Records that have a Sensor such that sensor.last_record_at == record.created ?

This way:

Records.objects.filter(sensor__last_record_at=F('created')).select_related('sensor')

would do the job.

This assumes you never have two records at the same time for a given sensor, otherwise, you could get several times the same Sensor , and that every Sensor has a Record with the last_record_at date.