简体   繁体   English

在BigQuery中填充连接表的缺失值

[英]Fill in missing values for joined tables in BigQuery

I have a table in BigQuery which contains net stock quantity for every day when some transaction occurs and table with price history, something like this: 我在BigQuery中有一个表,其中包含每次发生某些交易时的净库存数量以及包含价格历史的表格,如下所示:

WITH `trans` AS (
SELECT DATE '2018-10-02' trans_date, 10.0 quantity UNION ALL
SELECT DATE '2018-10-03', 5.0 UNION ALL
SELECT DATE '2018-10-05', 11.0 
),
`prices` AS (
SELECT DATE '2018-10-01' price_date, 1.0 price UNION ALL
SELECT DATE '2018-10-02', 2.0 UNION ALL
SELECT DATE '2018-10-03', 3.0 UNION ALL
SELECT DATE '2018-10-04', 4.0 UNION ALL
SELECT DATE '2018-10-05', 5.0 UNION ALL
SELECT DATE '2018-10-06', 6.0 UNION ALL
SELECT DATE '2018-10-07', 7.0 
)
SELECT 
price_date,
quantity, 
price
FROM (
SELECT price_date, quantity, price, trans_date FROM `trans`
RIGHT JOIN `prices`
ON trans.trans_date = prices.price_date
ORDER BY price_date
)

And I can't figure out how to fill in last known quantity where there is no transaction for that date, to get result something like this: 我无法弄清楚如何填写那个日期没有交易的最后已知数量,以获得如下结果:

price_date    quantity  price
2018-10-01    0.0    1.0
2018-10-02    10.0   2.0
2018-10-03    5.0    3.0
2018-10-04    5.0    4.0
2018-10-05    11.0   5.0
2018-10-06    11.0   6.0
2018-10-07    11.0   7.0

next step would be then to calculate value over time. 然后下一步将计算随时间变化的值。 I will be grateful for any example / suggestion how to do it. 我将很感激任何示例/建议如何做到这一点。

EDIT: I added stock symbols, to match more real example 编辑:我添加了股票代码,以匹配更多真实的例子

WITH `trans` AS (
SELECT DATE '2018-10-02' trans_date, 10.0 quantity, 'TX' symbol UNION ALL
SELECT DATE '2018-10-03' trans_date, 5.0 quantity, 'TX' UNION ALL
SELECT DATE '2018-10-05', 11.0, 'AX' 
),
`prices` AS (
 SELECT DATE '2018-10-01' price_date, 1.0 price, 'TX' symbol UNION ALL
  SELECT DATE '2018-10-02', 2.0, 'TX' UNION ALL
  SELECT DATE '2018-10-03', 3.0, 'TX' UNION ALL
  SELECT DATE '2018-10-04', 4.0, 'TX' UNION ALL
  SELECT DATE '2018-10-05', 5.0, 'TX' UNION ALL
  SELECT DATE '2018-10-06', 6.0, 'TX' UNION ALL
  SELECT DATE '2018-10-07', 7.0, 'TX' UNION ALL
  SELECT DATE '2018-10-08', 8.0, 'AX' UNION ALL
  SELECT DATE '2018-10-09', 9.0, 'TX' UNION ALL
  SELECT DATE '2018-10-10', 10.0, 'AX' UNION ALL
  SELECT DATE '2018-10-11', 11.0, 'TX' UNION ALL
  SELECT DATE '2018-10-12', 12.0, 'TX' 
)

SELECT
  price_date, 
  t.symbol AS symbol,
  IFNULL(
    ARRAY_AGG(
      IF(p.price_date >= t.trans_date AND p.symbol = t.symbol, quantity, NULL) 
      IGNORE NULLS ORDER BY trans_date DESC LIMIT 1
      )[OFFSET(0)],
  -1234567890) quantity,
  price
FROM `prices` p
CROSS JOIN `trans` t
GROUP BY price_date, price, symbol
HAVING quantity != -1234567890
ORDER BY price_date   

Below is for BigQuery Standard SQL 以下是BigQuery Standard SQL

#standardSQL
WITH `trans` AS (
  SELECT DATE '2018-10-02' trans_date, 10.0 quantity UNION ALL
  SELECT DATE '2018-10-03', 5.0 UNION ALL
  SELECT DATE '2018-10-05', 11.0 
),
`prices` AS (
  SELECT DATE '2018-10-01' price_date, 1.0 price UNION ALL
  SELECT DATE '2018-10-02', 2.0 UNION ALL
  SELECT DATE '2018-10-03', 3.0 UNION ALL
  SELECT DATE '2018-10-04', 4.0 UNION ALL
  SELECT DATE '2018-10-05', 5.0 UNION ALL
  SELECT DATE '2018-10-06', 6.0 UNION ALL
  SELECT DATE '2018-10-07', 7.0 
)
SELECT 
  price_date, 
  IFNULL(
    ARRAY_AGG(
      IF(p.price_date >= t.trans_date, quantity, NULL) 
      IGNORE NULLS ORDER BY trans_date DESC LIMIT 1
      )[OFFSET(0)],
  0) quantity,
  price
FROM `prices` p
CROSS JOIN `trans` t
GROUP BY price_date, price
-- ORDER BY price_date   

You can do: 你可以做:

select p.price_date,
       coalesce(t.quantity, lag(t.quantity ignore nulls) over (order by p.price_date)) as quantity
       p.price
from prices p join
     trans t
     on t.trans_date = p.price_date;

EDIT: 编辑:

That's right. 那就对了。 IGNORE NULL s works in some contexts but not others. IGNORE NULL在某些上下文中有效,但在其他情况下则无效。 Arrays are usually the fix in BigQuery: 数组通常是BigQuery中的修复:

select price_date,
       (select quantity
        from unnest(quantities) quantity with OFFSET n
        where quantity is not null
        order by n desc
        LIMIT 1
       ) as quantity,
       price
from (select p.price_date,
             array_agg(t.quantity) over (order by p.price_date) as quantities,
             p.price
      from prices p LEFT join
           trans t
           on t.trans_date = p.price_date
    ) pp;

声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM