通过Java应用程序连接到Spark SQL JDBC失败

Question

I have written a Java Application to perform SQL Queries on a hive-based database, using Spark SQL. However, connecting to the Spark SQL through JDBC from my Java Application gives me a connection refused error. The piece of code that I have written is below:

public static void main(String args[]) {
    Class.forName("org.apache.hive.jdbc.HiveDriver");
    conn = DriverManager.getConnection("jdbc:hive2://<ip>:10015/default","","");
    System.out.println("Connected!");
}

I am using HDP 2.5 on a Centos 7 OS inside a VMWare workstation on Windows 7 . The code is being run from Eclipse Neon, JDK 8 .

I am able to connect to port 10000 (for Hive) on the same IP from the code. Also, I have tried to make JDBC connections from Beeline to both 10000 and 10015 and both of them work properly. Beeline syntax is:

!connect jdbc:hive2://127.0.0.1:10015

The stack trace from the Java Code is:

1300 [main] INFO  org.apache.hive.jdbc.HiveConnection  - Could not open client transport with JDBC Uri: jdbc:hive2://192.168.225.128:10015/default
java.sql.SQLException: Could not open client transport with JDBC Uri: jdbc:hive2://192.168.225.128:10015/default: java.net.ConnectException: Connection refused: connect
at org.apache.hive.jdbc.HiveConnection.openTransport(HiveConnection.java:215)
at org.apache.hive.jdbc.HiveConnection.<init>(HiveConnection.java:163)
at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:105)
at java.sql.DriverManager.getConnection(DriverManager.java:664)
at java.sql.DriverManager.getConnection(DriverManager.java:247)

Answer 1

Have you stored the hive metadata in the mysql? Maybe the mysql server binds to localhost. Try to view /etc/mysql/mysql.conf.d/mysqld.cnf , and find this row:

bind-address = 127.0.0.1

remove it, and restart the mysql server.

Answer 2

I have managed to work it out, and the solution is fairly simple. You need to start Spark Thrift Servers to make this work.

Starting Spark Thrift Server using Ambari Web UI:

Just go to Ambari -> Spark. On the top-right corner, you can see options. Click on it and start Spark Thrift Servers from there.

In case you do not see Spark Thrift Servers in the Web UI, go to SPARK_HOME/sbin and type:

./start-thriftserver.sh --hiveconf hive.server2.thrift.port=10015

Wait for a minute or two after the server has started and you can make the JDBC connections successfully.