简体   繁体   English

雪花存储过程为外部表执行创建语句

[英]Snowflake Stored Procedure to execute create statement for external table

Trying to create a stored procedure in snowflake that takes arguments to generate 'create statement' for external table using metadata of the staged file from infer schema and executing it.尝试在雪花中创建一个存储过程,该过程需要 arguments 使用来自推断模式的暂存文件的元数据为外部表生成“创建语句”并执行它。 Since i am fairly new to Snowflake Stored procedure, would really appreciate inputs on how to achieve this?由于我对 Snowflake 存储过程相当陌生,因此真的很感谢有关如何实现这一点的输入吗?

CREATE OR REPLACE PROCEDURE DatabaseName.SchemaName.TEST_PROCEDURE(DB_NAME STRING, SCHEMA_NAME 
STRING, TABLE_NAME STRING , FILEPATH STRING)
returns string 
language javascript
as
$$
var cmd = `
   SELECT
   'CREATE OR REPLACE EXTERNAL TABLE ' || DB_NAME ||'.'||SCHEMA_NAME||'.'||TABLE_NAME|| ' (' 
||
LISTAGG(
  CLEAN_COLUMN || ' ' || TYPE || ' AS (' ||
  'VALUE:'|| CLEAN_COLUMN || '::' || TYPE || ')', ', ') AS QUERY_TEXT
 FROM (
SELECT *
,CASE WHEN COLUMN_NAME LIKE '%#%' THEN '"'||COLUMN_NAME||'"' ELSE COLUMN_NAME END AS CLEAN_COLUMN
from table(
  INFER_SCHEMA(
    LOCATION => '@db.src_xxx.stg_yyyy/2021-awdfg'
    ,FILE_FORMAT => '@db.src_xxx.parquet'
  )
) A
 )
`
var sql = snowflake.createStatement({sqlText: cmd});
var result = sql.execute();
    return 'Success';
$$;

The create external table supports USING TEMPLATE feature:创建外部表支持USING TEMPLATE功能:

CREATE EXTERNAL TABLE: Derive Column Definitions from Staged Files — Preview : 创建外部表:从暂存文件中导出列定义 - 预览

With this release, Snowflakes supports creating new external tables with the column definitions derived from a set of staged files containing semi-structured data.在此版本中, Snowflakes 支持使用从一组包含半结构化数据的暂存文件派生的列定义创建新的外部表。 Include the USING TEMPLATE keywords in your CREATE EXTERNAL TABLE statement在 CREATE EXTERNAL TABLE 语句中包含 USING TEMPLATE 关键字

CREATE OR REPLACE EXTERNAL TABLE myet USING TEMPLATE ( SELECT ARRAY_AGG(OBJECT_CONSTRUCT(*)) FROM TABLE( INFER_SCHEMA( LOCATION=>'@filestage', FILE_FORMAT=>'ffparquet' ) ) ) location=@filestage file_format=ffparquet auto_refresh=false;

This feature is currently limited to Apache Parquet, Apache Avro, and ORC files.此功能目前仅限于 Apache Parquet、Apache Avro 和 ORC 文件。

声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM