Django - How to combine two queries after doing a group by

Question

GOAL: Group by ITEM all INCOMING and its OUTGOING then SUM the quantity to come up with a simple table in my template like this:

Item        | Total In | Total Out| Stock
1. Cement   | 400      | 300      | 100

My models.py

class Incoming(models.Model):
    ...
    project_site = models.ForeignKey(ProjectSite, related_name='in_project_site', default=7, null=True, on_delete = models.SET_NULL)
    item = models.ForeignKey(Item, related_name='item_in', null=True, on_delete = models.SET_NULL)
    quantity = models.DecimalField('Quantity', db_index=True, max_digits=20, decimal_places=2, default=0)
    ...

class Outgoing(models.Model):
    ...
    base_in = models.ForeignKey('warehouse.Incoming', related_name='out', on_delete = models.SET_NULL, null=True)
    quantity = models.DecimalField('Quantity', db_index=True, max_digits=20, decimal_places=2, default=0)
    ...

Okay so the goal is to group all incoming by ITEM, then annotate the current total Incoming and Outgoing. So naturally I did this in my views.py

 incoming = Incoming.objects.filter(project_site__id=self.object.id)

 context['inventory'] =  incoming.values('item__item_name')\
    .annotate(tot_in=Sum('quantity'))\
    .annotate(tot_out=Sum('out__quantity'))

This does not give the proper SUM for the total_in, now after some search this is apparently an issue. One suggestion I found was to separate the query into to so this is what I did.

context['current_in'] =  incoming.values('item__item_name').annotate(tot_in=Sum('quantity'))
context['current_out'] =  incoming.values('item__item_name').annotate(tot_out=Sum('out__quantity'))

This gives me the correct SUM for each In and Out, now my problem is how to UNITE/COMBINE the two again so I could easily loop through it on my template? I did a search and tried the following to no luck.

list(chain(pagelist1, pagelist2))
# This does not combine items with same ITEM name. The output for this is
Item        | Total In | Total Out| Stock
Cement      | 400      |          | 
Cement      |          | 300      | 

doing a UNION doesn't help either. Any help would do.

Answer 1

How about this:

incoming.values('item__item_name').annotate(tot_in=Sum('quantity'), tot_out=Sum('out__quantity'))

If it does not work then but you can use pythonic approach here using zip_longest :

from itertools import zip_longest

current_in = incoming.values('item__item_name').annotate(tot_in=Sum('quantity'))
current_out = incoming.values('item__item_name').annotate(tot_out=Sum('out__quantity'))
context['current'] = [{**u, **v} for u, v in zip_longest(current_in, current_out, fillvalue={})]