识别配置单元表中缺少的分区

Question

My table runs daily and generates a partition cloumn called date on each day

For example: My query generates dates

2018-01-01
2018-01-02
2018-01-03
2018-01-06
2018-01-08

2018-01-05 & 2018-01-07 dates are missing. Is there any way to identify those missing dates?

Answer 1

Below queries will 1) create a temp table with sequential dates from start partition date to latest partition date 2) do a left join and see which partition dates are missing (partition_dt is null). Hope this helps. Thanks.

create table partition_dtes as 
with cal_date as (select min(partition_dt) as min_dt, max(partition_dt) as max_dt from mytable) 
select date_add(t.min_dt, pe.idx) as series_dte
from  cal_date t
lateral view
posexplode(split(space(datediff(t.max_dt,t.min_dt)),' ')) pe as idx, dte; 
Result: 
2018-01-01
2018-01-02
2018-01-03
2018-01-04
2018-01-05
2018-01-06
2018-01-07
2018-01-08

select distinct dte.series_dte
from partition_dtes dte
left join mytable  tbl
on dte.series_dte=tbl.partition_dt
where tbl.partition_dt is null
order by dte.series_dte;

Result:
   2018-01-04
   2018-01-05
   2018-01-07