[英]COPY INTO: is there a way to show number of records skipped during loading data into Snowflake?
I am using copy-into-table from an external location and there is an option to continue loading the data in case the row have corrupted data.我正在使用从外部位置复制到表中,并且可以选择继续加载数据,以防该行的数据损坏。 Is there an option to show how many rows were skipped while loading, like there is an option in Teradata TPT.
是否有一个选项可以显示加载时跳过了多少行,就像 Teradata TPT 中有一个选项一样。
Assuming that you are not doing transformations in your COPY INTO command, you can leverage the VALIDATE() function after the load and get the records skipped and the reason why they were not loaded:假设您没有在 COPY INTO 命令中进行转换,您可以在加载后利用 VALIDATE() function 并跳过记录以及未加载它们的原因:
https://docs.snowflake.com/en/sql-reference/functions/validate.html https://docs.snowflake.com/en/sql-reference/functions/validate.html
Example where t1 is your table being loaded. t1 是正在加载的表的示例。 You can also specify a specific query_id if you know it:
如果您知道,您还可以指定特定的 query_id:
select * from table(validate(t1, job_id => '_last'));
The COPY INTO outputs the following columns : COPY INTO 输出以下列:
ROWS_PARSED: Number of rows parsed from the source file
ROWS_LOADED: Number of rows loaded from the source file
ERROR_LIMIT: If the number of errors reaches this limit, then abort
ERRORS_SEEN: Number of error rows in the source file
The number of rows skipped can be calculated as ROWS_PARSED - ROWS_LOADED
.跳过的行数可以计算为
ROWS_PARSED - ROWS_LOADED
。 I am using pyodbc
the parsing of these columns might differ the way you are scripting.我正在使用
pyodbc
,这些列的解析可能与您编写脚本的方式不同。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.