如何在特定时间间隔内填充缺失值

Question

I have table in below format

user  timestamp              count  total_count

xyz   01-01-2020 00:12:00    45        45
xyz   01-01-2020 00:27:00    12        57
xyz   01-01-2020 00:29:00    11        68
xyz   01-01-2020 00:53:00    32        100

I want the data into 5 min interval like below (Expected Output)

user  timestamp              count  total_count

xyz   01-01-2020 00:05:00    0         0
xyz   01-01-2020 00:10:00    0         0
xyz   01-01-2020 00:15:00    45        45
xyz   01-01-2020 00:20:00    0         45
xyz   01-01-2020 00:25:00    0         45
xyz   01-01-2020 00:30:00    23        68
xyz   01-01-2020 00:35:00    0         68
xyz   01-01-2020 00:40:00    0         68
xyz   01-01-2020 00:45:00    0         68
xyz   01-01-2020 00:50:00    0         68
xyz   01-01-2020 00:55:00    32        100

I tried

   SELECT
        TIMESTAMP_SECONDS(5*60 * DIV(UNIX_SECONDS(timestamp), 5*60)) timekey,
        SUM(count) AS count,
        MAX(total_count) as total_count
   FROM db.table
   WHERE
        timestamp BETWEEN {{ start_date }}
        AND {{ end_date }}
        AND user = {{ user_id }}
   GROUP BY
        timekey
   ORDER BY
        timekey

Result of above query:

user  timestamp              count  total_count

xyz   01-01-2020 00:15:00    45        45
xyz   01-01-2020 00:30:00    23        68
xyz   01-01-2020 00:55:00    32        100

How can I fill those missing timestamps in above query and fill values of count(with zeros) and total_count(previous non null value)?

Answer 1

Use generate_timestamp_array() to fill in the missing values:

SELECT ts,
       SUM(t.count) AS count,
       MAX(t.total_count) as total_count
FROM UNNEST(GENERATE_TIMESTAMP_ARRAY( {{start_date}}, {{end_date}}, INTERVAL 5 minute)) ts LEFT JOIN
     db.table t
     ON t.timestamp >= ts AND
        t.timestamp < TIMESTAMP_ADD(ts, INTERVAL 5 minute) AND
        t.user = {{ user_id }}
GROUP BY ts
ORDER BY ts;

If you need to partition by the table, you can slightly modify the query:

SELECT ts,
       SUM(t.count) AS count,
       MAX(t.total_count) as total_count
FROM UNNEST(GENERATE_TIMESTAMP_ARRAY( {{start_date}}, {{end_date}}, INTERVAL 5 minute)) ts LEFT JOIN
     (SELECT t.*
      FROM db.table t
      WHERE timestamp BETWEEN {{ start_date }} AND {{ end_date }}
     ) t
     ON t.timestamp >= ts AND
        t.timestamp < TIMESTAMP_ADD(ts, INTERVAL 5 minute) AND
        t.user = {{ user_id }}
GROUP BY ts
ORDER BY ts;