如何在 BigQuery SQL 中选择记录子集？

Question

I have a set of records in BigQuery with a variable (CPIRating) that I would like to use to select a subset from.

CPIRating is an integer with a range from 0.1 to 250. I have over 10,000 records. What I am trying to create is a single subset/dataset of all the records where

1. It selects all records that have a CPIRating of 3.0 or greater
2. counts the number of records that have a CPIRating of 3.0 or greater
3. selects 4x that count of CPIRating 3.0 or greater and from the records that are less than 3.0 adds that number of records to the dataset, but does so from the lowest CPIRating value

As example, if the dataset has 1000 records with a CPIrating of 3.0 or greater, the query finds those, but also adds a further 4000 records (4x) that are below 3.0, but the 4000 records starts with the lowest CPIRating value (closest to 0.0) and adds those until it reaches the 4000.

Any ideas on how to structure that query in BigQuery?

Answer 1

First we generate some dummy data in table demo_tbl . Since CPIRating is normal distributed in this example, we choose values between zero and 3.2 as a maximum.

In the table help we calculate the rows, which have a CPIRating of 3 or higher. from demo_tbl,help joins both tables together and we obtain an additional column CPIRating_count . We numerate the rows by ascending CPIRating and create a row_number. Since this is a window function with over no where but a qualify clause is needed to filter the rows. In this filter the CPIRating<3.0 is not needed, but I find it easier to read.

With demo_tbl as (Select *, rand() *3.2 as CPIRating from unnest(generate_array(0,1*100)) id),
help as (select count(1) as CPIRating_count from demo_tbl where CPIRating>=3.0)

Select *,
row_number() over (order by CPIRating) as row_id
from demo_tbl,help
qualify (row_id < 4*help.CPIRating_count and CPIRating<3.0) or CPIRating>=3.0
order by row_id desc

The column CPIRating_count can also be generated by a window function instead of an join.