简体   繁体   English

如何基于S3分区数据在snowflake中创建外部表

[英]How to Create external table in snowflake based on S3 partitioned data

I have an s3 URL as s3://Sales//region.我有一个 s3 URL 作为 s3://Sales//region。

I have date buckets inside the region folder as:我在区域文件夹中有日期桶:

refdate=2022-02-26, refdate=2022-02-25 and refdate=2022-02-24参考日期=2022-02-26、参考日期=2022-02-25 和参考日期=2022-02-24

How would I create an external table in snowflake with partition as ref date folder?我如何在雪花中创建一个外部表并将分区作为参考日期文件夹? what have I tried so far:到目前为止我尝试了什么:

Create external table sales (..)
PARTITION BY refdate
with location = @sales_stage/region/
file_format = COMP_ap
aws_sns_topic='arn:aws:sns:us-west38:snowflake-dev-SNS'
auto_refresh = true ;
    

Thanks,谢谢,

Xi

create stage s2
  url='s3://Sales//region/'
  ...
  ;

create external table et1(
 date_part date as to_date(split_part(metadata$filename, '-', 3)
   || '/' || split_part(metadata$filename, '-', 4)
   || '/' || split_part(metadata$filename, '-', 5), 'YYYY/MM/DD'),
 timestamp bigint as (value:timestamp::bigint),
 partition by (date_part)
 location=@s2
 auto_refresh = true
 aws_sns_topic = 'arn:aws:sns:us-west38:snowflake-dev-SNS';

check the snowflake documentation here此处查看雪花文档

声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.

相关问题 PyArrow:如何将数据从 mongo 批处理到 S3 中的分区镶木地板 - PyArrow: How to batch data from mongo into partitioned parquet in S3 “分区数据”是什么意思 - S3 - what does it mean "partitioned data" - S3 如何使用 python 从 AWS S3 读取在列上分区的镶木地板文件数据 - How to read parquet file data partitioned on column from AWS S3 using python 如何将 SnowFlake S3 数据文件导出到我的 AWS S3? - How to export SnowFlake S3 data file to my AWS S3? 如何在分区数据上设置 Synapse Serverless SQL 外部表? - How do you setup a Synapse Serverless SQL External Table over partitioned data? 有没有最好的方法将数据从雪花获取到 s3 - Is there a best way to get data from snowflake to s3 允许 Snowflake 将数据卸载到 AWS S3 存储桶中 - Allowing Snowflake to unload data into AWS S3 bucket 将 Snowflake 中的数据作为 XLSX 文件保存到 S3 存储桶 - Saving data from Snowflake to S3 bucket as a XLSX file 无法在 aws EMR 集群中使用配置单元创建外部表,其中位置指向某个 S3 位置 - Unable to create external table using hive in aws EMR cluster where location pointing to some S3 location 即使 AWS s3 中没有文件,AWS Athena 创建外部表也会成功吗? - AWS Athena create external table succeeds even if AWS s3 doesn't have file in it?
 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM