简体   繁体   English

BigQuery,使用线性插值填充缺失值

[英]BigQuery, fill missing values with Linear interpolation

I have a table in Bigquery with data every 30 minutes, I want to show the data every 5 minutes, currently I am using this query to fill the null values with the existing values我在 Bigquery 中有一个每 30 分钟有数据的表,我想每 5 分钟显示一次数据,目前我正在使用此查询用现有值填充空值

SELECT
SETTLEMENTDATE,DUID,
LAST_VALUE(SCADAVALUE ignore nulls) OVER (
    PARTITION BY DUID ORDER BY SETTLEMENTDATE) AS SCADAVALUE from x

instead, is it possible to do Linear interpolation, something like this相反,是否可以进行线性插值,像这样

在此处输入图片说明

I have the column settlement date which is by 5 minutes, the column SCADAVALUEORIGIN Which has a value very 30 minutes, otherwise it is null, I want to add a column SCADAINTERPOLATION, which spread the values evenly between two the values of 30 minute, another issue is, as I refresh the data every 5 minutes, the last value will show null for (5,10,15,20,25) minutes, I hope, my explanation is clear我有 5 分钟的列结算日期,SCADAVALUEORIGIN 列的值非常 30 分钟,否则为空,我想添加一列 SCADAINTERPOLATION,它将值均匀分布在两个 30 分钟的值之间,另一个问题是,当我每 5 分钟刷新一次数据时,最后一个值将在 (5,10,15,20,25) 分钟内显示为空,我希望我的解释很清楚

I can speculate that you want something like this:我可以推测你想要这样的东西:

select timestamp_add(t.ts, interval min minute),
       (val * (30 - min) +
        lead(val) over (order by ts) * min
       ) / 30
from t cross join
     unnest(generate_array(0, 25, 5)) min;

Below is for BigQuery Standard SQL下面是 BigQuery 标准 SQL

#standardSQL
SELECT 
  TIMESTAMP_ADD(SETTLEMENTDATE, INTERVAL 5 * i MINUTE) AS SETTLEMENTDATE, 
  IF(i = 0, SCADAVALUEORIGIN, NULL) AS SCADAVALUEORIGIN,
  SCADAVALUEORIGIN AS SCADAVALUE,
  ROUND(SCADAVALUEORIGIN + IFNULL((next_value - SCADAVALUEORIGIN) / 6 * i, 0), 3) AS SCADAINTERPOLATION
FROM (
  SELECT SETTLEMENTDATE, SCADAVALUEORIGIN, 
    LEAD(SCADAVALUEORIGIN) OVER(ORDER BY SETTLEMENTDATE) next_value,
  FROM `project.dataset.table`
), UNNEST(GENERATE_ARRAY(0, 5)) i 

if to apply to sample data from your question - result is如果适用于您问题中的样本数据 - 结果是

在此处输入图片说明

声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM