Hive over Tez 通过 Hive JDBC - 错误

Question

I am using Hortonworks Hadoop HDP-2.3.2.0-2950 Hive over Tez engine

Below 2 queries are from Java code.

• select * from ascii -- works well
• select count(*) from ascii or select count(1) from ascii -- fails with error out

My code:

package com.hadoop.hive;

import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.PreparedStatement;
import java.sql.ResultSet;
import java.sql.SQLException;
/**
 * 
 * @author hdpadmin
 *
 */
public class HiveReadJDBCMain {
        private static String driverName = "org.apache.hive.jdbc.HiveDriver";

        /**
         * @param args
         * @throws SQLException
         */
        public static void main(String[] args) throws SQLException {
                Connection con = null;
                PreparedStatement pst = null;
                ResultSet  res = null;

                try {

                        // Load Hive Driver Clazz
                        Class.forName(driverName);
                        // No username and password required.(running in a local machine)
                        con = DriverManager.getConnection("jdbc:hive2://localhost:10000/labs");
                        //Connection con = DriverManager.getConnection("jdbc:hive2://localhost:10000/labs", "<Username>", "<Password>");
                        String tableName = "ascii";
                        // select * query
                        String sql = "select * from " + tableName;
                        System.out.println("Select Query Running: " + sql);
                        pst = con.prepareStatement(sql);
                        res = pst.executeQuery();
                        while (res.next()) {
                                System.out.println(String.valueOf(res.getInt(1)) + "\t" + res.getString(2));
                        }
                        pst.close();
                        res.close();

                        // regular hive query
                        sql = "select count(*) from " + tableName;
                        System.out.println("Count Query Running: " + sql);
                        pst = con.prepareStatement(sql);
                        res = pst.executeQuery();
                        while (res.next()) {
                                System.out.println(res.getString(1));
                        }
                } catch (ClassNotFoundException e) {
                        e.printStackTrace();
                        System.exit(1);
                } catch (Exception e) {
                        e.printStackTrace();
                        System.exit(1);
                } finally {
                        res.close();
                        pst.close();
                        con.close();
                }
        }
}

Error
java.sql.SQLException: Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.tez.TezTask
        at org.apache.hive.jdbc.HiveStatement.execute(HiveStatement.java:282)
        at org.apache.hive.jdbc.HiveStatement.executeQuery(HiveStatement.java:378)
        at org.apache.hive.jdbc.HivePreparedStatement.executeQuery(HivePreparedStatement.java:109)
        at com.hadoop.hive.HiveReadJDBCMain.main(HiveReadJDBCMain.java:48)

Answer 1

是的，我已经通过使用我的用户名传递以下连接对象进行了修复。谢谢....:)

con = DriverManager.getConnection("jdbc:hive2://localhost:10000/labs", "hdpadmin","");

Answer 2

i also fixed by passing username in connection string . Thanks :)

Answer 3

The queries like Select * from table; work well because they don't launch any MR job. When an MR job is launched, the user with which the JDBC Connection was made requires having proper access to HDFS directories which the Job will require. If you want to see the actual stack trace run the same job by running in MR mode. To do so pass the following in the Connection String or as a parameter:-

connection = DriverManager.getConnection("jdbc:hive2://localhost:10000/labs?hive.execution.engine=mr", "hive","");

This would print the actual error, I guess Tez abstracts away the actual error.

This answer is in response to Saurab's comment on the answer above. Didn't have enough repo to comment :p.

Answer 4

In case your using the AWS Hive JDBC to talk with a default EMR cluster the following setup in Intellij worked for me

The trick was using user hadoop