[英]Teradata JDBC Error 1338 after certain number of batch executions
I am trying to insert 3.8M records into a Teradata table containing 14 columns, through a batch of 10000 records using RJDBC
package of R. It always crashes after inserting 380000 records ie at 39th batch execution.我正在尝试使用 R 的
RJDBC
包通过一批 10000 条记录向包含 14 列的 Teradata 表中插入 3.8M 条记录。插入 380000 条记录后,即在第 39 次批量执行时,它总是崩溃。 This is the error I get,这是我得到的错误,
Error in .jcall(ps, "[I", "executeBatch") :
.jcall(ps, "[I", "executeBatch") 中的错误:
java.sql.BatchUpdateException: [Teradata JDBC Driver] [TeraJDBC 15.10.00.22] [Error 1338] [SQLState HY000] A failure occurred while executing a PreparedStatement batch request.java.sql.BatchUpdateException: [Teradata JDBC Driver] [TeraJDBC 15.10.00.22] [Error 1338] [SQLState HY000] 执行 PreparedStatement 批处理请求时发生故障。 Details of the failure can be found in the exception chain that is accessible with getNextException.
可以在可通过 getNextException 访问的异常链中找到失败的详细信息。
I've tried changing to a different set of rows from the same dataframe.我尝试从同一数据帧更改为一组不同的行。 And, behavior remains same, crashes at the 39th batch execution.
并且,行为保持不变,在第 39 批执行时崩溃。
Any idea what is this Error 1338
and what could be the fix?知道这个
Error 1338
是什么以及修复方法是什么? Also, it mentions about getNextException
but how do I use that with R?此外,它提到了
getNextException
但我如何将它与 R 一起使用?
The approach I'm using is similar to this one https://developer.teradata.com/blog/ulrich/2013/11/a-wider-test-case-on-r-jdbc-fastload我使用的方法类似于这个https://developer.teradata.com/blog/ulrich/2013/11/a-wider-test-case-on-r-jdbc-fastload
To get the initial exception "object", you need to use .jgetEx to and then you can use the getNextException method on it to retrieve the root cause.要获取初始异常“对象”,您需要使用 .jgetEx to 然后您可以使用 getNextException 方法来检索根本原因。
See this link on the teradata developers forum for a complete example.有关完整示例,请参阅 teradata 开发人员论坛上的此链接。
Typical causes of an exception during a batch insert are:批量插入期间异常的典型原因是:
Since you say that you've tested different datasets and they all fail at the same execution, that's very likely a " no more room in database ".由于您说您已经测试了不同的数据集并且它们都在同一次执行中失败,这很可能是“数据库中没有更多空间”。
However, this doesn't necessarily mean that the space that has been granted to you is full: it might be that the table has a poorly chosen Primary Index (or the data has an error that produces duplicate primary index ), resulting in a very skewed distribution.但是,这并不一定意味着授予您的空间已满:可能是该表的主索引选择不当(或数据有错误导致重复的主索引),导致非常偏态分布。 In this case, you would be wasting a lot of space that could be reclaimed if you can define a more apt primary index (or use a NOPI table, if you're using it as some sort of a stage table).
在这种情况下,如果您可以定义更合适的主索引(或使用 NOPI 表,如果您将其用作某种阶段表),您将浪费大量可以回收的空间。
Here you can find a discussion on this "space waste" topic with lots of useful queries to diagnose the issue. 在这里,您可以找到有关“空间浪费”主题的讨论,其中包含许多有用的查询来诊断问题。
Same happens to me I was trying to insert 27.4M records into a Teradata table containing 8 columns.同样发生在我身上,我试图将 27.4M 记录插入包含 8 列的 Teradata 表中。 I've got the same error at 5.35M row.
我在 5.35M 行有同样的错误。
The only thing that works for me was make a commit every 1M records.唯一对我有用的是每 1M 条记录提交一次。
So few notes here.所以这里的笔记很少。
Thread.sleep
with single seconds.Thread.sleep
。 This will trigger GC.17.00.00.03
.17.00.00.03
发现了一些竞争条件。 For performance purpose it is reusing old objects with a weak reference to something to be collected.
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.