蜂巢索引无法重建-DAG失败到顶点失败

Question

我在HDP2.2上使用Hive14，但在Hive上建立索引存在问题。 我可以创建一个索引。

create INDEX ix_key ON TABLE DbTest.Tbl_test(TEST_KEY)
as 'org.apache.hadoop.hive.ql.index.compact.CompactIndexHandler' WITH DEFERRED REBUILD;

之后，我将数据加载到表中并建立索引。

ALTER INDEX ix_key ON DbTest.Tbl_test REBUILD;

Hive建立了索引，并且工作正常，性能提高了。 现在，我想重建索引，但是总是出现错误：

INFO  : Session is already open
INFO  : Tez session was closed. Reopening...
INFO  : Session re-established.
INFO  : 

ERROR : Status: Failed
ERROR : Vertex failed, vertexName=Map 1, vertexId=vertex_1426585957958_2810_1_00, diagnostics=[Vertex vertex_1426585957958_2810_1_00 [Map 1] killed/failed due to:ROOT_INPUT_INIT_FAILURE, Vertex Input: Tbl_test initializer failed, vertex=vertex_1426585957958_2810_1_00 [Map 1], java.lang.NullPointerException
at org.apache.hadoop.hive.ql.exec.tez.DynamicPartitionPruner.initialize(DynamicPartitionPruner.java:135)
at org.apache.hadoop.hive.ql.exec.tez.DynamicPartitionPruner.prune(DynamicPartitionPruner.java:100)
at org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator.initialize(HiveSplitGenerator.java:109)
at org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable$1.run(RootInputInitializerManager.java:245)
at org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable$1.run(RootInputInitializerManager.java:239)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable.call(RootInputInitializerManager.java:239)
at org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable.call(RootInputInitializerManager.java:226)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
]
ERROR : Vertex killed, vertexName=Reducer 2, vertexId=vertex_1426585957958_2810_1_01, diagnostics=[Vertex received Kill in INITED state., Vertex vertex_1426585957958_2810_1_01 [Reducer 2] killed/failed due to:null]
ERROR : DAG failed due to vertex failure. failedVertices:1 killedVertices:1
Error: Error while processing statement: FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.tez.TezTask (state=08S01,code=2)

基表存在，我可以对其进行查询。 索引表也存在。 如果我在另一个表上创建新索引并运行rebuild-command，则会收到相同的错误。 我用beeline和CLI尝试了-错误始终相同。

希望有人对如何解决这个问题有一些想法。

Answer 1

这似乎在动态分区修剪器中失败，您可以使用“ hive.tez.dynamic.partition.pruning = false”将其关闭。 您可能还需要考虑在此处提交错误： https ： //issues.apache.org