Fill in gaps in time series after a group by date in BigQuery

Question

I have a big dataset with multiple timeseries of different products and quantities sold. After I grouped by Product_ID and Date(monthly), products that have not been sold during a specific month generate gaps in the time series. How could I fill all these gaps with rows with quantity sold 0?

I found similar stuff but haven't been able to adapt it to my situation

Answer 1

Your question is rather vague, best to provide example input and outputs in future, but here is my guess as to what you are trying to do.

You will probably want to create a calendar table, with the intervals of your data. I will use daily as the example as you have not suggested, you can adapt for different regularities as required.

CREATE OR REPLACE TABLE `project.my_dataset.daily_calendar` (
   MY_DATE          DATE        NOT NULL
)


INSERT INTO `project.my_dataset.daily_calendar`
SELECT
MY_DATE
FROM
(
SELECT DATE_ADD('2000-01-01',INTERVAL param DAY) AS MY_DATE
FROM unnest(GENERATE_ARRAY(0, 10000, 1)) as param
)
;

The above will create a calendar with daily intervals from 10000 days from the year 2000, adjust according to your requirements.

You will then want to cross join your data with the calendar data, this will create a record for every day, and every product.

select
cal.date,
your_data.product_name,
ifnull(your_data.quantity, 0) -- if no data then make the value 0
from `project.dataset.table`
cross join
`project.my_dataset.daily_calendar`

From your vague description, this could work. Ideally provide example input and output in future to get a more accurate response.

Edit:

An alternate (arguably better) method to fill the calendar table, as suggested by @Cylldby:

INSERT INTO `project.my_dataset.daily_calendar`
SELECT
MY_DATE
FROM
(
SELECT param AS MY_DATE
FROM unnest(GENERATE_DATE_ARRAY('2010-01-01', '2030-12-31')) as param
)
;