Which Django ORM or raw SQL I should write to get exactly what I need

Question

I'm using Postgresql. I have model:

class EventsList(CreatedUpdatedMixin):
    start = models.DateTimeField()
    end = models.DateTimeField()
    is_inner = models.BooleanField()

Assume that I have those DB entries:

start	end	is_inner
2021-12-09 14:30:12	2021-12-09 15:00:21	true
2021-12-09 14:00:05	2021-12-10 21:00:15	false
2021-12-10 09:00:39	2021-12-10 09:30:50	true
2021-12-10 14:00:00	2021-12-11 15:00:00	true
2021-12-14 10:00:00	2021-12-14 11:00:00	true
2021-12-13 13:30:00	2021-12-16 14:30:00	false
2021-12-14 13:10:00	2021-12-15 00:30:00	true
2021-12-14 10:30:00	2021-12-16 13:34:00	false
2021-12-15 13:30:00	2021-12-15 18:30:00	true

And there is the result I need:

[
    {"2021-12-09": {"external_events": 1, "internal_events": 1}},
    {"2021-12-10": {"external_events": 0, "internal_events": 2}},
    {"2021-12-11": {"external_events": 0, "internal_events": 1}},
    {"2021-12-13": {"external_events": 1, "internal_events": 0}},
    {"2021-12-14": {"external_events": 2, "internal_events": 2}},
    {"2021-12-15": {"external_events": 2, "internal_events": 2}},
    {"2021-12-16": {"external_events": 2, "internal_events": 0}},
]

So I want to get all existing dates and for each date get the count of external events (where is_inner == False) and the count of internal events (where is_inner == True). How can I do it with Django ORM or in raw SQL? For now, I come up with

EventsList.objects.annotate(
    start_day=Cast("start", output_field=DateField())
).values("start_day").annotate(
    external_events=Count("id", filter=Q(is_inner=False)),
    internal_events=Count("id", filter=Q(is_inner=True)),
).values(
    "start_day", "external_events", "internal_events"
)

This code returns an almost right result (but only for "start" dates). I need to include all dates (start date, end date, and all dates between them). Will appreciate any help.

Answer 1

First, to get all dates that are not in your db you'd loop between the max and min date to get all dates and for each date run queries to match the filters you want.

Answer 2

Using raw SQL, first expand the list of dates using generate_series and lateral join then do conditional aggregation. So here it is, a bit verbose but I hope easily readable. SQL Fiddle

with t as
(
 select is_inner, d::date from the_table 
 cross join lateral generate_series
 (
  date_trunc('day', "start"), 
  date_trunc('day', "end"),  
  interval '1 day'
 ) as d
)
select d as event_date, 
       count(*) filter (where not is_inner) external_events,   
       count(*) filter (where is_inner) internal_events
from t
group by d order by d; 

You can shape the JSON structure in the question as well using jsonb_build_object like this:

with t as
(
 select is_inner, d::date from the_table 
 cross join lateral generate_series
 (
  date_trunc('day', "start"), 
  date_trunc('day', "end"),  
  interval '1 day'
 ) as d
)
select jsonb_build_object
(
  d::text, 
  jsonb_build_object('external_events', count(*) filter (where not is_inner),   
                     'internal_events', count(*) filter (where is_inner))
) as date_info   
from t
group by d order by d; 

date_info
{"2021-12-09": {"external_events": 1, "internal_events": 1}}
{"2021-12-10": {"external_events": 1, "internal_events": 2}}
{"2021-12-11": {"external_events": 0, "internal_events": 1}}
{"2021-12-13": {"external_events": 1, "internal_events": 0}}
{"2021-12-14": {"external_events": 2, "internal_events": 2}}
{"2021-12-15": {"external_events": 2, "internal_events": 2}}
{"2021-12-16": {"external_events": 2, "internal_events": 0}}