[英]Hive queries failing on Tez but succeeding on Map-Reduce when connecting from Beeline
I am running into a weird error.我遇到了一个奇怪的错误。 I am running a simple select * query with a where clause, following is the summary of query execution status
我正在运行一个带有 where 子句的简单 select * 查询,以下是查询执行状态的摘要
I need to solve for point 3. This is the error trace I am getting and unable to find what the root cause of this failure is and what this error log is trying to convey.我需要解决第 3 点。这是我得到的错误跟踪,无法找到此故障的根本原因是什么以及此错误日志试图传达什么。
at org.apache.hive.service.cli.operation.Operation.toSQLException(Operation.java:380)
at org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:257)
at org.apache.hive.service.cli.operation.SQLOperation.access$800(SQLOperation.java:91)
at org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork$1.run(SQLOperation.java:348)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1840)
at org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork.run(SQLOperation.java:362)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)' SQL<select `ID`, `ISDELETED`, `ACCOUNTID`, `CREATEDBYID`, `CREATEDDATE`, `FIELD`, `OLDVALUE`, `NEWVALUE`, `AUDIT_UPD_TS`, `SRC_OP_TYP`, `GG_INGEST_TS` from `t4i_ent_sfdc_b2b_psa`.`sf_accounthistory` x WHERE SRC_OP_TYP='NA'>```
I was able to solve this.我能够解决这个问题。 The problem was I was connecting my application to Hive via JDBC without specifying a user.
问题是我在没有指定用户的情况下通过 JDBC 将我的应用程序连接到 Hive。 For queries where simple streaming of data was required, it was succeeding, but where Map-Reduce jobs were being triggered to write to HDFS, the writing operation was failing with the error
对于需要简单数据流的查询,它是成功的,但是在触发 Map-Reduce 作业以写入 HDFS 的情况下,写入操作失败并出现错误
Failed to execute tez graph.
org.apache.hadoop.security.AccessControlException: Permission denied: user=anonymous, access=WRITE, inode="/user":hdfs:hadoop:drwxr-xr-x
To resolve this, I added the user=hadoop;为了解决这个问题,我添加了 user=hadoop; in the JDBC URL and the queries run fine now.
在 JDBC URL 中,查询现在运行良好。
Try invoking beeline
( Tez engine
) as below and then run your query:尝试如下调用
beeline
( Tez engine
),然后运行您的查询:
beeline -u "jdbc:hive2://<host>:<port>,/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2-batch?tez.queue.name=<yarn-queue-name>"
If above doesn't work then try to fix any issue in SQL.如果上述方法不起作用,请尝试修复 SQL 中的任何问题。 I see 'x' before Where clause in your sql query, that may be the issue.
我在 sql 查询中的 Where 子句之前看到“x”,这可能是问题所在。 Try to remove that and run your query.
尝试删除它并运行您的查询。
`sf_accounthistory` x WHERE SRC_OP_TYP='NA'
Hope this is helpful希望这会有所帮助
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.