针对聚合列的Hive查询优化在select语句中出现一次

Question

if there are multi aggregated column in one select, would the be evaluated only once? for example:

select
    date,
    count(userid) as uv,
    sum(isclick) as clickcnt,
    count(userid) / sum(isclick) as ctr
from
    user_access_log
group by
    1

here both count(userid) and sum(isclick) are used twice, would they be evaluated twice or only once, will hive do any query optimization?

Answer 1

This is too long for a comment.

It doesn't make a difference. The expense of running an aggregation query is almost entirely in bringing the rows for groups together. For the most part, the aggregations themselves are not expensive.

The one exception is count(distinct) (well, distinct with any form). This requires a bunch more overhead.

If you really want to run the aggregations only once, you can use a subquery:

select ual.*, (uv / clickcnt) as ctr
from (select date, count(userid) as uv, sum(isclick) as clickcnt,
      from user_access_log
      group by 1
     ) ual;

To be honest, I suspect that you actually want count(distinct userid) , so this might give a small improvement in performance.