Django: Selecting queryset fields based on a condition

Question

Folks,

Here's the schema of the table T (Let's say Model name is also T with fields A, B and C):

 A    B   C
---  --- ---
 1    a   b
 2    b   NULL
 3    c   c

Now, I want to select values (A, C or B). I can select A and C as follows:

 T.objects.all().values_list('A','C')

But it will give me A and C values only. All I want is if C is NULL, then I should get B. I don't know whether I can achieve this directly using any condition or join (Sorry, I'm not a SQL guy) but I can definitely achieve this using follows:

 [(x,z or y) for (x,y,z) in T.objects.all().values_list('A','B','C')]

So, there are two open questions:

1. Is there any better way of doing it? (Preferably using Queryset/Django ORM functions)

2. In case of tens of thousands of records, what is the memory efficient / optimized way of doing this? Is there any way other than limiting the queryset into smaller chunks (let's say 1000 records at once) using the loop (assuming ordered primary key) as follows:

    max_pkey = T.objects.all().order_by('-pk')[0] current = 0 chunk = 1000 while current <= max_pkey: objs = T.objects.filter(pk__gt=current)[:chunk] Process the objects 

Answer 1

In answer to your second question for very large data sets you may want to look at the iterator:

https://docs.djangoproject.com/en/dev/ref/models/querysets/#django.db.models.query.QuerySet.iterator

As this won't try and load the entire data set at time of evaluation, but obviously there are trade offs.

Answer 2

I think I can answer the first question.

1. Is there any better way of doing it? (Preferably using Queryset/Django ORM functions)

Django provides Conditional expressions that lets you do this.

From Django docs,

Conditional expressions let you use if … elif … else logic within filters, annotations, aggregations, and updates. A conditional expression evaluates a series of conditions for each row of a table and returns the matching result expression.

For the schema you provided, you can use this:

T.objects.annotate(colA = F('A'), 
                   colB_C = Case(When(C__isnull=True, then=F('B')),
                                 default=F('C')))

For every row, this selects the value in column B if the value in column C is null. Otherwise, value in column C is selected.