[英]Combining several datasets in a Netezza System
We have a database with 65 tables that begin with Mon_Tues_Wed_201701 through Mon_Tues_Wed_202205.我们有一个包含 65 个表的数据库,从 Mon_Tues_Wed_201701 到 Mon_Tues_Wed_202205。 We create new tables with the naming convention of Mon_Tues_Wed_yyyymm so obviously it's growing.
我们使用 Mon_Tues_Wed_yyyymm 的命名约定创建新表,因此显然它正在增长。 Each table contains about 1m-2m rows.
每个表包含大约 1m-2m 行。
Here's what we want to do:这是我们想要做的:
We are currently doing this in SAS on a Grid System:我们目前在网格系统上的 SAS 中执行此操作:
Unfortunately this process takes 6 hours (1+ min per view import for a total time of 65+ min, 15 min to add the variables, 2+ hours to export to Netezza).不幸的是,这个过程需要 6 小时(每次视图导入 1 分钟以上,总时间 65 分钟以上,添加变量需要 15 分钟,导出到 Netezza 需要 2 小时以上)。
If you can show me code that would do the above, I'd really be grateful.如果您可以向我展示可以执行上述操作的代码,我将不胜感激。 I'm new to Netezza and it seems many of the rules from T-SQL processing do not apply.
我是 Netezza 的新手,似乎 T-SQL 处理中的许多规则都不适用。
Thank you谢谢
Paula保拉
-- This can be done dynamically via a Stored Procedure
-- For example ...
create or replace procedure my_dynamic_sql()
returns integer
language nzplsql
execute as caller
as begin_proc
DECLARE
tables RECORD;
my_sql VARCHAR;
union_statement VARCHAR;
newline VARCHAR;
BEGIN
newline := chr(10);
union_statement := '';
my_sql := 'create table MON_TUES_WED_SUMMARY as' || newline ;
raise notice 'Generating dynamic sql for';
FOR tables IN
SELECT TABLENAME FROM _V_TABLE WHERE OBJTYPE = 'TABLE' AND upper(TABLENAME) LIKE 'MON_TUES_WED_______' ORDER BY 1
LOOP
raise notice ' Table: %', tables.tablename;
my_sql := my_sql || union_statement || 'select ''' || tables.tablename || '''::varchar(128) as source_table, * from ' || tables.tablename ;
union_statement := ' union all' || newline ;
END LOOP;
raise notice 'Dropping the OLD results table';
execute immediate 'DROP TABLE MON_TUES_WED_SUMMARY IF EXISTS;';
raise notice 'The SQL looks like this %', my_sql;
raise notice 'Executing the sql now ...';
execute immediate my_sql;
raise notice 'Done creating table MON_TUES_WED_SUMMARY';
end;
end_proc;
--------------------------------------------------------------------------------
-- Then, to invoke the stored procedure
call my_dynamic_sql;
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.