How to apply: count(distinct ...) over (partition by ... order by) in big query?

Question

I currently have this source table .

I am trying to get this second table from the first table, in SQL on GCP BigQuery.

My Query is the following :

        SELECT
            SE.MARKET_ID,
            SE.LOCAL_POS_ID,
            SE.BC_ID,
            LEFT(SE.SALE_CREATION_DATE,6) AS DATE_ID_MONTH,

            COUNT(DISTINCT
                CASE
                    WHEN FLAG
                    THEN SE.CUST_ID
                END)
            OVER (PARTITION BY SE.MARKET_ID, SE.LOCAL_POS_ID, SE.BC_ID, LEFT(SE.SALE_CREATION_DATE,4) ORDER BY LEFT(SE.SALE_CREATION_DATE,6)) AS NB_ACTIVE_CUSTOMERS

        FROM
            SE
        GROUP BY
            SE.MARKET_ID, SE.LOCAL_POS_ID, SE.BC_ID, LEFT(SE.SALE_CREATION_DATE,6)

However, I get this error that I did not succeed to bypass :

Window ORDER BY is not allowed if DISTINCT is specified at [12:107]

I can't create a previous table with the following request :

SELECT DISTINCT
        SE.MARKET_ID,
        SE.LOCAL_POS_ID,
        SE.BC_ID,
        LEFT(SE.SALE_CREATION_DATE,6) AS DATE_ID_MONTH,
        CASE
            WHEN FLAG
            THEN SE.CUST_ID
            ELSE NULL
        END AS VALID_CUST_ID
FROM
        SE

in order to use a dense_rank() after that because I have 50 others indicators (and 500M rows) to add to this table (indicators based on other flags) and I can't obviously create a WITH for each of them, I need to have it in only a few WITH or none (exactly like my current query is supposed to do).

Has anyone got a clue on how I can handle that please ?

Answer 1

You mention using dense_rank() later but wouldn't dense_rank() work for this counter also?

DENSE_RANK(CASE WHEN FLAG = 1 THEN SE.CUST_ID END)
    OVER (
        PARTITION BY SE.MARKET_ID, SE.LOCAL_POS_ID, SE.BC_ID, LEFT(SE.SALE_CREATION_DATE, 4)
        ORDER BY LEFT(SE.SALE_CREATION_DATE, 6)
    ) AS NB_ACTIVE_CUSTOMERS

Answer 2

Consider below approach

select * except(ids), 
  array_length(array(
    select distinct id
    from unnest(split(ids)) id
  )) as nb_active_customers, 
  format('%t', array(
    select distinct id
    from unnest(split(ids)) id
  )) as distinct_values
from (
  select market_id, local_pos_id, bc_id, date_id_month,
    string_agg('' || ids) over(partition by market_id order by date_id_month) ids
  from (
    select market_id, local_pos_id, bc_id, left(sale_creation_date,6) AS date_id_month,
      string_agg('' || cust_id) ids
    from se
    where flag = 1
    group by market_id, local_pos_id, bc_id, date_id_month
  )
) t          

if applied to sample data in your question - output is