简体   繁体   English

BigQuery - BI Engine 衡量节省

[英]BigQuery - BI Engine measuring savings

I have deployed BI Engine in one of my Google projects and I am measuring cost savings using the following query我已经在我的一个 Google 项目中部署了 BI 引擎,我正在使用以下查询来衡量成本节省

with tbl
as
(
    select creation_time, total_bytes_processed, total_bytes_billed,    
           5 * (total_bytes_processed / 1000000000000) as cost_projected, 
           5 * (total_bytes_billed / 1000000000000) as cost_actual

      from `region-us`.INFORMATION_SCHEMA.JOBS_BY_PROJECT b
     where 1=1
       and job_type = "QUERY"
       and creation_time >= '2022-05-10 11:30:00.000 UTC'
       and creation_time <= '2022-05-10 19:00:00.000 UTC'
)
select sum(cost_projected) - sum(cost_actual) as savings
  from tbl
 where 1=1
;

However, I noticed that very often I have accelerated queries (bi_engine_statistics.bi_engine_mode = 'FULL') for which 'total_bytes_billed = total_bytes_processed'.但是,我注意到我经常加速查询 (bi_engine_statistics.bi_engine_mode = 'FULL'),其中 'total_bytes_billed = total_bytes_processed'。 I was expecting that for accelerated queries total_bytes_billed should be equal to zero which does not seem to be the case.我期望对于加速查询 total_bytes_billed 应该等于零,但似乎并非如此。

So the questions are:所以问题是:

  1. Is my query the correct way of measuring savings,我的查询是衡量储蓄的正确方法吗?
  2. Is it normal to have fully accelerated queries with total_bytes_billed > 0?使用 total_bytes_billed > 0 进行完全加速查询是否正常?
  1. To answer your question on calculation of savings :回答有关储蓄计算的问题:
    I think it is the correct way of measuring your savings, but some queries of type QUERY cannot be used for BI Engine, so it's somewhat unfair to keep counting them in.我认为这是衡量节省的正确方法,但是一些QUERY类型的查询不能用于 BI Engine,因此继续计算它们有点不公平。
    Which is why I wrote the script in this SO question:这就是我在这个 SO 问题中编写脚本的原因:
    BigQuery BI Engine: how to choose a good reservation size? BigQuery BI Engine:如何选择合适的预留大小?
    Also you could improve on converting bytes to TB by calculating as follows:您还可以通过如下计算改进将字节转换为 TB:
    sum(total_bytes_processed) / pow(1024, 4) AS TB_processed

  1. As for your question on FULL mode and still having to pay :至于您关于 FULL 模式但仍需付费的问题:
    I have turned BI Engine on and if I run this query on my data, I get 0 results, so all my bi_engine_mode='FULL' queries have savings:我已经打开 BI 引擎,如果我对我的数据运行这个查询,我得到 0 个结果,所以我所有的bi_engine_mode='FULL'查询都节省了:
SELECT 
    total_bytes_processed, 
    total_bytes_billed, 
    bi_engine_statistics
FROM `my_project_id.region-eu.INFORMATION_SCHEMA.JOBS`
WHERE 1=1
    and bi_engine_statistics.bi_engine_mode = 'FULL'
    and total_bytes_processed = total_bytes_billed
    and total_bytes_processed > 0

声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM