PostgreSQL get relative average with group by

Question

I have a table as follows. The rows are in a specific order.

id    |      value
------+---------------------
 1    |        2
 1    |        4     
 1    |        3
 2    |        2
 2    |        2
 2    |        5

I would want to group the rows by the column 'id' and get the average of value displayed in each column in terms of the previous values of the column (As explained in the following example within brackets)

id    |      value  |    RelativeAverage    
------+-------------+--------------------
 1    |        2    |        (2/1) = 2
 1    |        4    |        (2+4 /2) = 3
 1    |        3    |        (2+4+3 / 3) = 3
 2    |        2    |        (2/1) = 2
 2    |        2    |        (2+2 / 2) = 2
 2    |        5    |        (2+2+5 / 3) = 9

Is there an approach with which I can achieve this?

Thanks in Advance

Answer 1

Wrong query:

select 
  id, value, 

  sum(value) over(arrangement), rank() over(arrangement),

  sum(value) over(arrangement)::numeric / rank() over(arrangement) 
  as relative_average
from tbl
window arrangement as (partition by id order by id);

Output (wrong):

| id | value | sum | rank | relative_average |
|----|-------|-----|------|------------------|
|  1 |     2 |   9 |    1 |                9 |
|  1 |     4 |   9 |    1 |                9 |
|  1 |     3 |   9 |    1 |                9 |
|  2 |     1 |   8 |    1 |                8 |
|  2 |     2 |   8 |    1 |                8 |
|  2 |     5 |   8 |    1 |                8 |

You need something that sorts correctly in order for sum and rank to work properly on your actual arrangement of your data. You can use table row's hidden ctid field, but that is Postgres-specific

Correct query:

select 
    id, value, 

    sum(value) over(arrangement), rank() over(arrangement),

    sum(value) over(arrangement)::numeric / rank() over(arrangement) 
    as relative_average
from tbl
window arrangement as (partition by id order by tbl.ctid);

Output (correct):

| id | value | sum | rank |   relative_average |
|----|-------|-----|------|--------------------|
|  1 |     2 |   2 |    1 |                  2 |
|  1 |     4 |   6 |    2 |                  3 |
|  1 |     3 |   9 |    3 |                  3 |
|  2 |     1 |   1 |    1 |                  1 |
|  2 |     2 |   3 |    2 |                1.5 |
|  2 |     5 |   8 |    3 | 2.6666666666666665 |

Best way is to introduce a serial primary key, so doing a running-total( sum over() ) based on actual arrangement of your data could be achieved.

CREATE TABLE tbl
    (ordered_pk serial primary key, "id" int, "value" int)
;

INSERT INTO tbl
    ("id", "value")
VALUES
    (1, 2),
    (1, 4),
    (1, 3),
    (2, 1),
    (2, 2),
    (2, 5)
;

Correct query:

select 
    id, value, 

    sum(value) over(arrangement), rank() over(arrangement),

    sum(value) over(arrangement)::numeric / rank() over(arrangement) 
    as relative_average
from tbl
window arrangement as (partition by id order by ordered_pk);

Output (correct):

| id | value | sum | rank |   relative_average |
|----|-------|-----|------|--------------------|
|  1 |     2 |   2 |    1 |                  2 |
|  1 |     4 |   6 |    2 |                  3 |
|  1 |     3 |   9 |    3 |                  3 |
|  2 |     1 |   1 |    1 |                  1 |
|  2 |     2 |   3 |    2 |                1.5 |
|  2 |     5 |   8 |    3 | 2.6666666666666665 |

Live test: http://sqlfiddle.com/#!17/f18276/1

You can order by value , but it will yield different result, not necessarily wrong output, but different because of different arrangement of values. And then you also need to use row_number instead of rank / dense_rank due to possible duplication of values. Here I made an example of duplicate values.

Correct query:

select 
    id, value, 

    sum(value) over(arrangement),

    row_number() over(arrangement),
    rank() over(arrangement),  
    dense_rank() over(arrangement),    

    sum(value) over(arrangement)::numeric / row_number() over(arrangement) 
    as relative_average
from tbl
window arrangement as (partition by id order by value)

Output:

| id | value | sum | row_number | rank | dense_rank |   relative_average |
|----|-------|-----|------------|------|------------|--------------------|
|  1 |     2 |   2 |          1 |    1 |          1 |                  2 |
|  1 |     3 |   5 |          2 |    2 |          2 |                2.5 |
|  1 |     4 |   9 |          3 |    3 |          3 |                  3 |
|  2 |     1 |   1 |          1 |    1 |          1 |                  1 |
|  2 |     2 |   5 |          2 |    2 |          2 |                2.5 |
|  2 |     2 |   5 |          3 |    2 |          2 | 1.6666666666666667 |
|  2 |     5 |  10 |          4 |    4 |          3 |                2.5 |

Live test: http://sqlfiddle.com/#!17/2b5aac/1

Answer 2

Not so proud of my other answer

Just use avg .

Today I Learned rows between unbounded preceding and current row . And it works with the actual arrangement of data even in the absence of a good candidate field for order by . It looks like that at least you can get away with using Postgres' hidden ctid field, or you can even avoid using serial primary. Recommending though to use serial primary key or date created field to order by upon.

Here's a better query. No need to divide, just use avg

select 
    id, value, 

    avg(value) over(arrangement rows between unbounded preceding and current row)
from tbl
window arrangement as (partition by id);

Output

| id | value |                avg |
|----|-------|--------------------|
|  1 |     2 |                  2 |
|  1 |     4 |                  3 |
|  1 |     3 |                  3 |
|  2 |     1 |                  1 |
|  2 |     2 |                1.5 |
|  2 |     5 | 2.6666666666666665 |

select 
    id, value, 

    sum(value) over(arrangement), rank() over(arrangement),

    sum(value) over(arrangement)::numeric / rank() over(arrangement) 
    as relative_average,    

    avg(value) over(arrangement rows between unbounded preceding and current row)
from tbl
window arrangement as (partition by id order by id);

Output:

| id | value | sum | rank | relative_average |                avg |
|----|-------|-----|------|------------------|--------------------|
|  1 |     2 |   9 |    1 |                9 |                  2 |
|  1 |     4 |   9 |    1 |                9 |                  3 |
|  1 |     3 |   9 |    1 |                9 |                  3 |
|  2 |     1 |   8 |    1 |                8 |                  1 |
|  2 |     2 |   8 |    1 |                8 |                1.5 |
|  2 |     5 |   8 |    1 |                8 | 2.6666666666666665 |

select 
    id, value, 

    sum(value) over(arrangement), rank() over(arrangement),

    sum(value) over(arrangement)::numeric / rank() over(arrangement) 
    as relative_average,

    avg(value) over(arrangement rows between unbounded preceding and current row)    
from tbl
window arrangement as (partition by id order by tbl.ctid);

Output:

| id | value | sum | rank |   relative_average |                avg |
|----|-------|-----|------|--------------------|--------------------|
|  1 |     2 |   2 |    1 |                  2 |                  2 |
|  1 |     4 |   6 |    2 |                  3 |                  3 |
|  1 |     3 |   9 |    3 |                  3 |                  3 |
|  2 |     1 |   1 |    1 |                  1 |                  1 |
|  2 |     2 |   3 |    2 |                1.5 |                1.5 |
|  2 |     5 |   8 |    3 | 2.6666666666666665 | 2.6666666666666665 |

select 
    id, value, 

    sum(value) over(arrangement), rank() over(arrangement),

    sum(value) over(arrangement)::numeric / rank() over(arrangement) 
    as relative_average,

    avg(value) over(arrangement rows between unbounded preceding and current row)    
from tbl
window arrangement as (partition by id order by ordered_pk);

Output:

| id | value | sum | rank |   relative_average |                avg |
|----|-------|-----|------|--------------------|--------------------|
|  1 |     2 |   2 |    1 |                  2 |                  2 |
|  1 |     4 |   6 |    2 |                  3 |                  3 |
|  1 |     3 |   9 |    3 |                  3 |                  3 |
|  2 |     1 |   1 |    1 |                  1 |                  1 |
|  2 |     2 |   3 |    2 |                1.5 |                1.5 |
|  2 |     5 |   8 |    3 | 2.6666666666666665 | 2.6666666666666665 |

Live test: http://sqlfiddle.com/#!17/f18276/9

rows between unbounded preceding and current row can be also written as rows unbounded preceding http://sqlfiddle.com/#!17/f18276/11

---

And here's the result with order by value when value have duplicates.

select 
    id, value, 

    sum(value) over(arrangement),

    row_number() over(arrangement) as rn,
    rank() over(arrangement) as rank,  
    dense_rank() over(arrangement) drank,    

    trunc( sum(value) over(arrangement)::numeric 
        / row_number() over(arrangement), 2) as ra__rn,
    trunc( sum(value) over(arrangement)::numeric 
        / row_number() over(arrangement), 2) as ra__rank,    
    trunc( sum(value) over(arrangement)::numeric 
        / row_number() over(arrangement), 2) as ra__drank,        

    trunc( avg(value) over(arrangement 
    rows between unbounded preceding and current row), 2) as ra
from tbl
window arrangement as (partition by id order by value)

Output:

| id | value | sum | rn | rank | drank | ra__rn | ra__rank | ra__drank |   ra |
|----|-------|-----|----|------|-------|--------|----------|-----------|------|
|  1 |     2 |   2 |  1 |    1 |     1 |      2 |        2 |         2 |    2 |
|  1 |     3 |   5 |  2 |    2 |     2 |    2.5 |      2.5 |       2.5 |  2.5 |
|  1 |     4 |   9 |  3 |    3 |     3 |      3 |        3 |         3 |    3 |
|  2 |     1 |   1 |  1 |    1 |     1 |      1 |        1 |         1 |    1 |
|  2 |     2 |   5 |  2 |    2 |     2 |    2.5 |      2.5 |       2.5 |  1.5 |
|  2 |     2 |   5 |  3 |    2 |     2 |   1.66 |     1.66 |      1.66 | 1.66 |
|  2 |     5 |  10 |  4 |    4 |     3 |    2.5 |      2.5 |       2.5 |  2.5 |

Live test: http://sqlfiddle.com/#!17/2b5aac/16

And here's the result with order by ordered_pk when value have duplicates.

select 
    id, value,

    sum(value) over(arrangement),

    row_number() over(arrangement) as rn,
    rank() over(arrangement) as rank,  
    dense_rank() over(arrangement) drank,    

    trunc( sum(value) over(arrangement)::numeric 
        / row_number() over(arrangement), 2) as ra__rn,
    trunc( sum(value) over(arrangement)::numeric 
        / row_number() over(arrangement), 2) as ra__rank,    
    trunc( sum(value) over(arrangement)::numeric 
        / row_number() over(arrangement), 2) as ra__drank,        

    trunc( avg(value) over(arrangement 
    rows between unbounded preceding and current row), 2) as ra
from tbl
window arrangement as (partition by id order by ordered_pk)

| id | value | sum | rn | rank | drank | ra__rn | ra__rank | ra__drank |   ra |
|----|-------|-----|----|------|-------|--------|----------|-----------|------|
|  1 |     2 |   2 |  1 |    1 |     1 |      2 |        2 |         2 |    2 |
|  1 |     4 |   6 |  2 |    2 |     2 |      3 |        3 |         3 |    3 |
|  1 |     3 |   9 |  3 |    3 |     3 |      3 |        3 |         3 |    3 |
|  2 |     1 |   1 |  1 |    1 |     1 |      1 |        1 |         1 |    1 |
|  2 |     2 |   3 |  2 |    2 |     2 |    1.5 |      1.5 |       1.5 |  1.5 |
|  2 |     2 |   5 |  3 |    3 |     3 |   1.66 |     1.66 |      1.66 | 1.66 |
|  2 |     5 |  10 |  4 |    4 |     4 |    2.5 |      2.5 |       2.5 |  2.5 |

Live test: http://sqlfiddle.com/#!17/baaf9/2

Answer 3

If I assume that you have an ordering column in the table, then what you want is:

select t.*,
       avg(value) over (partition by id
                        order by ?
                        rows between unbounded preceding and current row
                       ) as running_avg
from t;

The ? is the ordering column.

In other words, Postgres has a single built-in function that does exactly what you want -- and the function happens to be standard SQL.

The window frame using rows is required, because the default is range .

If you do not have an ordering column, then you should add one. I strongly advise you to NOT use ctid for this purpose. It might seem like it works on small sets of data, but it is not stable over time and it might not work on larger sets of data.

If you expect your data to be ordered by inserts, then use a serial column to capture the insert order.