How to fill Null with the previous value in PostgreSQL?

Question

I have a table which contains Null values. I need to replace them with a previous non-Null value. This is an example of data which I have:

   date    | category | start_period | period_number |
------------------------------------------------------
2018-01-01 |    A     |       1      |       1       |
2018-01-02 |    A     |       0      |      Null     |
2018-01-03 |    A     |       0      |      Null     |
2018-01-04 |    A     |       0      |      Null     |
2018-01-05 |    B     |       1      |       2       |
2018-01-06 |    B     |       0      |      Null     |
2018-01-07 |    B     |       0      |      Null     |
2018-01-08 |    A     |       1      |       3       |
2018-01-09 |    A     |       0      |      Null     |
2018-01-10 |    A     |       0      |      Null     |

The result should look like this:

   date    | category | start_period | period_number |
------------------------------------------------------
2018-01-01 |    A     |       1      |       1       |
2018-01-02 |    A     |       0      |       1       |
2018-01-03 |    A     |       0      |       1       |
2018-01-04 |    A     |       0      |       1       |
2018-01-05 |    B     |       1      |       2       |
2018-01-06 |    B     |       0      |       2       |
2018-01-07 |    B     |       0      |       2       |
2018-01-08 |    A     |       1      |       3       |
2018-01-09 |    A     |       0      |       3       |
2018-01-10 |    A     |       0      |       3       |

I tried the following query, but in this case, only the first Null value will be replaced.

select 
date,
category,
start_period,
case
    when period_number isnull then lag(period_number) over()
    else period_number
end as period_number
from period_table;

Also, I tried to use first_value() window function, but I don't know how to set up the correct window.

Any help is highly appreciated.

Answer 1

If you replace your case statement with:

(
    select
        _.period_number
    from
        period_table as _
    where
        _.period_number is not null
        and _.category = period_table.category
        and _.date <= period_table.date
    order by
        _.date desc
    limit 1
) as period_number

Then it should have the intended effect. It's nowhere near as elegant as a window function but I don't think window functions are quite flexible enough for your specific use case here (Or at least, if they are, I don't know how to flex them that much)

Answer 2

You can join table with itself and get desired value. Assuming your date column is the primary key or unique.

update your_table upd set period_number = tbl.period_number 
from
(
   select b.date, max(b2.date) as d2 from your_table b 
   inner join d_batch_tab b2 on b2.date< b.date and b2.period_number  is not null 
   group by b.date
)t 
inner join your_table tbl on tbl.date = t.d2
where t.date= upd.date

If you don't need to update the table but only a select statement then

select yt.date, yt.category, yt.start_period, tbl.period_number
from your_table yt
inner join 
(
   select b.date, max(b2.date) as d2 from your_table b 
   inner join d_batch_tab b2 on b2.date< b.date and b2.period_number  is not null 
   group by b.date
)t on yt.date = t.date
inner join your_table tbl on tbl.date = t.d2

Answer 3

Examples of windows function and frame clause:

    select 
    date,category,score
    ,FIRST_VALUE(score) OVER (
        PARTITION BY category
        ORDER BY date RANGE BETWEEN UNBOUNDED 
        PRECEDING AND CURRENT ROW
    ) as last_score
from testing.rec_test
order by date, category


select 
    date,category,score
    ,LAST_VALUE(score) OVER (
        PARTITION BY category
        ORDER BY date RANGE BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
    ) as last_score
from testing.rec_test
order by date, category