简体   繁体   中英

Snowflake COPY command directory read

相关查询的图像

COPY INTO SALES.SF_DEV.CUSTOMER_PARQUET_TGT FROM @EXTERNAL_STAGE_PARQUET_DEMO/partioned/customer/Country=INDIA/ MATCH_BY_COLUMN_NAME = CASE_SENSITIVE FILE_FORMAT =  (TYPE = PARQUET);

The above command will load all the files recursively from dir Country=INDIA and all its sub-directories.

Que: Do we have an option to control recursive file loading, what I mean to load only files from Country=INDIA not from its sub-directories?

PS This image is with the pattern suggested by @gokhan

Can you try to use the pattern option?

COPY INTO SALES.SF_DEV.CUSTOMER_PARQUET_TGT 
FROM @EXTERNAL_STAGE_PARQUET_DEMO/partioned/customer/Country=INDIA/
MATCH_BY_COLUMN_NAME = CASE_SENSITIVE FILE_FORMAT =  (TYPE = PARQUET)
PATTERN = '[^\/]*';

The pattern is to prevent any sub-directories (files containing the slash characters).

My test environment:

ls @my_stage/test/Country=INDIA/; 

+--------------------------------------------------------------------------+
|                                   name                                   |
+--------------------------------------------------------------------------+
| s3://mysecretbucketname/test/test/Country=INDIA/data_0_0_0.csv.gz        |
| s3://mysecretbucketname/test/test/Country=INDIA/hododo/data_0_0_0.csv.gz |
+--------------------------------------------------------------------------+

ls @my_stage/test/Country=INDIA/ pattern = '[^\/]*';

+-------------------------------------------------------------------+
|                               name                                |
+-------------------------------------------------------------------+
| s3://mysecretbucketname/test/test/Country=INDIA/data_0_0_0.csv.gz |
+-------------------------------------------------------------------+

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM