[英]BigQuery: Loop through tables in dataset and drop tables with a specific prefix
Our development environment in BigQuery is isolated to a development dataset ie dev
in BigQuery.我们在 BigQuery 中的开发环境与开发数据集隔离,即 BigQuery 中的
dev
。 The environments are further isolated by a prefix of the ticket for each table ie DATA-100-change-table
would correspond to the DATA-100 ticket.环境通过每个表的票证前缀进一步隔离,即
DATA-100-change-table
将对应于 DATA-100 票证。
I am aware of setting TTL for BigQuery , however, I am also interested in having a query I could run by hand to delete the tables.我知道为 BigQuery 设置 TTL ,但是,我也对可以手动运行以删除表的查询感兴趣。
So far, I have the below:到目前为止,我有以下内容:
begin
-- Create temporary table of tables to drop from `dev` with prefix
create temp table to_drop (drop_string STRING)
as
(
select concat("drop table if exists ", table_catalog, ".", table_schema, ".", table_name)
from dev.INFORMATION_SCHEMA.TABLES
where table_name like "DATA-100%"
);
-- Loop through table and execute drop_string statements
for drop_statement in (select drop_string from to_drop)
do
execute immediate drop_statement;
end for;
end
However, this fails with the following error:但是,这失败并出现以下错误:
Query error: Cannot coerce expression drop_statement to type STRING at [14:23]
Is my approach right here?我的方法就在这里吗? How do I best delete all tables with a prefix in BigQuery?
如何最好地删除 BigQuery 中带有前缀的所有表?
Also, if possible, I would like this query to handle views as well.另外,如果可能的话,我希望这个查询也能处理视图。
The variable drop_statement in the for loop contains a struct. for 循环中的变量 drop_statement 包含一个结构。 So you have to access the string with
drop_statement.drop_string
.所以你必须使用
drop_statement.drop_string
访问字符串。
begin
-- Create temporary table of tables to drop from `dev` with prefix
create temp table to_drop (drop_string STRING)
as
(
select concat("drop table if exists ", table_catalog, ".", table_schema, ".", table_name)
from dev.INFORMATION_SCHEMA.TABLES
where table_name like "DATA-100%" and table_type = 'BASE TABLE'
);
-- Loop through table and execute drop_string statements
for drop_statement in (select drop_string from to_drop)
do
execute immediate drop_statement.drop_string;
end for;
end
To drop VIEWs, just replace table_type = "BASE TABLE"
with table_type = "VIEW"
and use drop view if exists
instead.要删除视图,只需将
table_type = "BASE TABLE"
替换为table_type = "VIEW"
并使用drop view if exists
)。
Using p13rr0m's answer as inspiration, I've come up with the below:以p13rr0m 的回答为灵感,我得出了以下结论:
begin
create temp table to_drop (drop_string STRING)
as
(
select
case
when table_type = "BASE TABLE" then concat("drop table if exists `", table_catalog, ".", table_schema, ".", table_name, "`")
when table_type = "VIEW" then concat("drop view if exists `", table_catalog, ".", table_schema, ".", table_name, "`")
end as drop_string
from
`dev.INFORMATION_SCHEMA.TABLES`
where
table_name like "DATA_100%"
);
for drop_statement in (select drop_string from to_drop)
do
execute immediate drop_statement.drop_string;
end for;
end
Had to add backticks to the tables as it was failing on dropping views.不得不在表中添加反引号,因为它无法删除视图。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.