堆棧內存溢出
  簡體   English   中英

無法從主機連接到ZooKeeper / Hive到沙盒Hortonworks HDP VM

[英]Can not connect to ZooKeeper/Hive from host to Sandbox Hortonworks HDP VM

 原文  2018-01-16 16:33:38  8   1    hadoop/ jdbc/ hive/ hortonworks-data-platform/ hortonworks-sandbox

問題描述

不久前，我下載了HDP-Sandbox（在Oracle VirtualBox VM中），但從未使用過，現在我正嘗試使用Hive HDBC從外部訪問數據。

我使用從mvnrepository獲得的apache中的hive-jdbc 1.2.2，並將所有依賴項放在類路徑中，或者從其存儲庫中獲得了hortonworks JDBC 

http://mvnrepository.com/artifact/org.apache.hive/hive-jdbc/1.2.2

要么 

http://repo.hortonworks.com/content/repositories/releases/org/apache/hive/hive-jdbc/1.2.1000.2.6.4.0-91

我寫了這個非常基本的程序 

import java.sql.Connection;
import java.sql.DriverManager;

public class Main {

    public static void main(final String[] args) {

        try {
            final Class<?> driver = Class.forName("org.apache.hive.jdbc.HiveDriver");
            DriverManager.setLoginTimeout(10);
            try (final Connection connection = DriverManager.getConnection("jdbc:hive2://sandbox.hortonworks.com:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2")) {
                //
            }
        } catch (final Exception e) {
            e.printStackTrace();
        }
    }
}

連接失敗，並出現Null Pointer Exception或超時（請參見下面的輸出日志，一個關於我使用的每個JDBC驅動程序的日志），我想這是由配置錯誤引起的-即使我在...框中使用了沙箱。

我的沙盒版本 

Sandbox information:
Created on: 25_10_2016_08_11_26 for
Hadoop stack version:  Hadoop 2.7.3.2.5.0.0-1245
Ambari Version: 2.4.0.0-1225
Ambari Hash: 59175b7aa1ddb74b85551c632e3ce42fed8f0c85
Ambari build:  Release : 1225
Java version:  1.8.0_111
OS Version:  CentOS release 6.8 (Final)

我的虛擬機屏幕 

HDP 2.5
http://hortonworks.com
To initiate your Hortonworks Sandbox session,
please open a browser and enter this address
in the browser's address field:
http://127.0.0.1:8888/

我的主機文件： 

…/…
127.0.0.1 sandbox.hortonworks.com
…/…

Ambari屏幕上的配置單元摘要 

Hive Metastore        Started No alerts
HiveServer2           Started No alerts
WebHCat Server        Started No alerts
Hive Client           1 Hive Client Installed
HiveServer2 JDBC URL  jdbc:hive2://sandbox.hortonworks.com:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2

Zookeeper摘要 

ZooKeeper Server    Started No alerts
ZooKeeper Client    1 ZooKeeper Client Installed

我被困住了，任何幫助將不勝感激！

謝謝 ！

程序輸出：

對於apache配置單元jdbc 1.2.2： 

16:52:06,378  INFO hive.jdbc.Utils: Supplied authorities: sandbox.hortonworks.com:2181
16:52:06,500  INFO curator.framework.imps.CuratorFrameworkImpl: Starting
16:52:06,503 DEBUG curator.CuratorZookeeperClient: Starting
16:52:06,503 DEBUG curator.ConnectionState: Starting
16:52:06,503 DEBUG curator.ConnectionState: reset
16:52:06,515  INFO zookeeper.ZooKeeper: Client environment:zookeeper.version=3.4.6-1569965, built on 02/20/2014 09:09 GMT
16:52:06,515  INFO zookeeper.ZooKeeper: Client environment:java.version=9.0.1
16:52:06,515  INFO zookeeper.ZooKeeper: Client environment:java.vendor=Oracle Corporation
16:52:06,515  INFO zookeeper.ZooKeeper: Client environment:java.home=D:\JDK\jdk9
16:52:06,516  INFO zookeeper.ZooKeeper: Initiating client connection, connectString=sandbox.hortonworks.com:2181 sessionTimeout=60000 watcher=org.apache.curator.ConnectionState@24269709
16:52:06,521 DEBUG zookeeper.ClientCnxn: zookeeper.disableAutoWatchReset is false
16:52:06,631  INFO zookeeper.ClientCnxn: Opening socket connection to server 127.0.0.1/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error)
16:52:06,633  INFO zookeeper.ClientCnxn: Socket connection established to 127.0.0.1/127.0.0.1:2181, initiating session
16:52:06,635 DEBUG zookeeper.ClientCnxn: Session establishment request sent on 127.0.0.1/127.0.0.1:2181
16:52:06,645  INFO zookeeper.ClientCnxn: Session establishment complete on server 127.0.0.1/127.0.0.1:2181, sessionid = 0x160ff8bf98b0008, negotiated timeout = 40000
16:52:06,650  INFO curator.framework.state.ConnectionStateManager: State change: CONNECTED
16:52:06,663 DEBUG zookeeper.ClientCnxn: Reading reply sessionid:0x160ff8bf98b0008, packet:: clientPath:null serverPath:null finished:false header:: 1,12  replyHeader:: 1,1347,0  request:: '/hiveserver2,F  response:: v{'serverUri=sandbox.hortonworks.com:10000;version=1.2.1000.2.5.0.0-1245;sequence=0000000007},s{467,467,1477381899524,1477381899524,0,15,0,0,10,1,1340} 
16:52:06,669 DEBUG zookeeper.ClientCnxn: Reading reply sessionid:0x160ff8bf98b0008, packet:: clientPath:null serverPath:null finished:false header:: 2,4  replyHeader:: 2,1347,0  request:: '/hiveserver2/serverUri=sandbox.hortonworks.com:10000;version=1.2.1000.2.5.0.0-1245;sequence=0000000007,F  response:: #686976652e736572766572322e61757468656e7469636174696f6e3d4e4f4e453b686976652e736572766572322e7472616e73706f72742e6d6f64653d62696e6172793b686976652e736572766572322e7468726966742e7361736c2e716f703d617574683b686976652e736572766572322e7468726966742e62696e642e686f73743d73616e64626f782e686f72746f6e776f726b732e636f6d3b686976652e736572766572322e7468726966742e706f72743d31303030303b686976652e736572766572322e7573652e53534c3d66616c7365,s{1340,1340,1516116649382,1516116649382,0,0,0,99360168454324228,213,0,1340} 
16:52:06,670  INFO hive.jdbc.ZooKeeperHiveClientHelper: Selected HiveServer2 instance with uri: hive.server2.authentication=NONE;hive.server2.transport.mode=binary;hive.server2.thrift.sasl.qop=auth;hive.server2.thrift.bind.host=sandbox.hortonworks.com;hive.server2.thrift.port=10000;hive.server2.use.SSL=false
16:52:06,670 DEBUG curator.framework.imps.CuratorFrameworkImpl: Closing
16:52:06,671 DEBUG curator.CuratorZookeeperClient: Closing
16:52:06,671 DEBUG curator.ConnectionState: Closing
16:52:06,672 DEBUG zookeeper.ZooKeeper: Closing session: 0x160ff8bf98b0008
16:52:06,672 DEBUG zookeeper.ClientCnxn: Closing client for session: 0x160ff8bf98b0008
16:52:06,674 DEBUG zookeeper.ClientCnxn: Reading reply sessionid:0x160ff8bf98b0008, packet:: clientPath:null serverPath:null finished:false header:: 3,-11  replyHeader:: 3,1348,0  request:: null response:: null
16:52:06,674 DEBUG zookeeper.ClientCnxn: Disconnecting client for session: 0x160ff8bf98b0008
16:52:06,674  INFO zookeeper.ZooKeeper: Session: 0x160ff8bf98b0008 closed
16:52:06,674  INFO hive.jdbc.Utils: Resolved authority: hive.server2.authentication=NONE;hive.server2.transport.mode=binary;hive.server2.thrift.sasl.qop=auth;hive.server2.thrift.bind.host=sandbox.hortonworks.com;hive.server2.thrift.port=10000;hive.server2.use.SSL=false
16:52:06,675  INFO zookeeper.ClientCnxn: EventThread shut down
16:52:06,731  INFO hive.jdbc.HiveConnection: Will try to open client transport with JDBC Uri: jdbc:hive2://hive.server2.authentication=NONE;hive.server2.transport.mode=binary;hive.server2.thrift.sasl.qop=auth;hive.server2.thrift.bind.host=sandbox.hortonworks.com;hive.server2.thrift.port=10000;hive.server2.use.SSL=false/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2
16:52:06,733 DEBUG thrift.transport.TSaslTransport: opening transport org.apache.thrift.transport.TSaslClientTransport@49049a04
java.lang.NullPointerException
    at org.apache.thrift.transport.TSocket.open(TSocket.java:170)
    at org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:266)
    at org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:37)
    at org.apache.hive.jdbc.HiveConnection.openTransport(HiveConnection.java:206)
    at org.apache.hive.jdbc.HiveConnection.<init>(HiveConnection.java:178)
    at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:105)
    at java.sql/java.sql.DriverManager.getConnection(DriverManager.java:678)
    at java.sql/java.sql.DriverManager.getConnection(DriverManager.java:252)
    at com.mycompany.hadoop.hiveconnect.Main.main(Main.java:13)

使用Hortonworks配置單元jdbc 1.2.1000.2.6.4.0-91 

16:49:57,414  INFO hive.jdbc.Utils: Supplied authorities: sandbox.hortonworks.com:2181
16:49:57,544  INFO curator.framework.imps.CuratorFrameworkImpl: Starting
16:49:57,546 DEBUG curator.CuratorZookeeperClient: Starting
16:49:57,546 DEBUG curator.ConnectionState: Starting
16:49:57,546 DEBUG curator.ConnectionState: reset
16:49:57,555  INFO zookeeper.ZooKeeper: Client environment:zookeeper.version=3.4.6-91--1, built on 01/04/2018 10:14 GMT
16:49:57,555  INFO zookeeper.ZooKeeper: Client environment:java.version=9.0.1
16:49:57,555  INFO zookeeper.ZooKeeper: Client environment:java.vendor=Oracle Corporation
16:49:57,555  INFO zookeeper.ZooKeeper: Client environment:java.home=D:\JDK\jdk9
16:49:57,556  INFO zookeeper.ZooKeeper: Initiating client connection, connectString=sandbox.hortonworks.com:2181 sessionTimeout=60000 watcher=org.apache.curator.ConnectionState@1190200a
16:49:57,560 DEBUG zookeeper.ClientCnxn: zookeeper.disableAutoWatchReset is false
16:49:57,668  INFO zookeeper.ClientCnxn: Opening socket connection to server 127.0.0.1/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error)
16:49:57,673  INFO zookeeper.ClientCnxn: Socket connection established, initiating session, client: /127.0.0.1:17993, server: 127.0.0.1/127.0.0.1:2181
16:49:57,674 DEBUG zookeeper.ClientCnxn: Session establishment request sent on 127.0.0.1/127.0.0.1:2181
16:49:57,787  INFO zookeeper.ClientCnxn: Session establishment complete on server 127.0.0.1/127.0.0.1:2181, sessionid = 0x160ff8bf98b0006, negotiated timeout = 40000
16:49:57,807  INFO curator.framework.state.ConnectionStateManager: State change: CONNECTED
16:49:58,378 DEBUG zookeeper.ClientCnxn: Reading reply sessionid:0x160ff8bf98b0006, packet:: clientPath:null serverPath:null finished:false header:: 1,12  replyHeader:: 1,1343,0  request:: '/hiveserver2,F  response:: v{'serverUri=sandbox.hortonworks.com:10000;version=1.2.1000.2.5.0.0-1245;sequence=0000000007},s{467,467,1477381899524,1477381899524,0,15,0,0,10,1,1340} 
16:49:58,400 DEBUG zookeeper.ClientCnxn: Reading reply sessionid:0x160ff8bf98b0006, packet:: clientPath:null serverPath:null finished:false header:: 2,4  replyHeader:: 2,1343,0  request:: '/hiveserver2/serverUri=sandbox.hortonworks.com:10000;version=1.2.1000.2.5.0.0-1245;sequence=0000000007,F  response:: #686976652e736572766572322e61757468656e7469636174696f6e3d4e4f4e453b686976652e736572766572322e7472616e73706f72742e6d6f64653d62696e6172793b686976652e736572766572322e7468726966742e7361736c2e716f703d617574683b686976652e736572766572322e7468726966742e62696e642e686f73743d73616e64626f782e686f72746f6e776f726b732e636f6d3b686976652e736572766572322e7468726966742e706f72743d31303030303b686976652e736572766572322e7573652e53534c3d66616c7365,s{1340,1340,1516116649382,1516116649382,0,0,0,99360168454324228,213,0,1340} 
16:49:58,402 DEBUG curator.framework.imps.CuratorFrameworkImpl: Closing
16:49:58,403 DEBUG curator.CuratorZookeeperClient: Closing
16:49:58,403 DEBUG curator.ConnectionState: Closing
16:49:58,405 DEBUG zookeeper.ZooKeeper: Closing session: 0x160ff8bf98b0006
16:49:58,405 DEBUG zookeeper.ClientCnxn: Closing client for session: 0x160ff8bf98b0006
16:49:58,466 DEBUG zookeeper.ClientCnxn: Reading reply sessionid:0x160ff8bf98b0006, packet:: clientPath:null serverPath:null finished:false header:: 3,-11  replyHeader:: 3,1344,0  request:: null response:: null
16:49:58,466 DEBUG zookeeper.ClientCnxn: Disconnecting client for session: 0x160ff8bf98b0006
16:49:58,466  INFO zookeeper.ZooKeeper: Session: 0x160ff8bf98b0006 closed
16:49:58,467  INFO hive.jdbc.Utils: Resolved authority: sandbox.hortonworks.com:10000
16:49:58,467  INFO zookeeper.ClientCnxn: EventThread shut down
16:49:58,545 DEBUG thrift.transport.TSaslTransport: opening transport org.apache.thrift.transport.TSaslClientTransport@2794eab6
16:49:58,549 DEBUG thrift.transport.TSaslClientTransport: Sending mechanism name PLAIN and initial response of length 20
16:49:58,550 DEBUG thrift.transport.TSaslTransport: CLIENT: Writing message with status START and payload length 5
16:49:58,551 DEBUG thrift.transport.TSaslTransport: CLIENT: Writing message with status COMPLETE and payload length 20
16:49:58,551 DEBUG thrift.transport.TSaslTransport: CLIENT: Start message handled
16:49:58,551 DEBUG thrift.transport.TSaslTransport: CLIENT: Main negotiation loop complete
16:49:58,551 DEBUG thrift.transport.TSaslTransport: CLIENT: SASL Client receiving last message
16:50:08,552  WARN hive.jdbc.HiveConnection: Failed to connect to sandbox.hortonworks.com:10000
16:50:08,553  INFO curator.framework.imps.CuratorFrameworkImpl: Starting
16:50:08,554 DEBUG curator.CuratorZookeeperClient: Starting
16:50:08,554 DEBUG curator.ConnectionState: Starting
16:50:08,554 DEBUG curator.ConnectionState: reset
16:50:08,554  INFO zookeeper.ZooKeeper: Initiating client connection, connectString=sandbox.hortonworks.com:2181 sessionTimeout=60000 watcher=org.apache.curator.ConnectionState@46cdf8bd
16:50:08,557  INFO zookeeper.ClientCnxn: Opening socket connection to server 127.0.0.1/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error)
16:50:08,558  INFO zookeeper.ClientCnxn: Socket connection established, initiating session, client: /127.0.0.1:18018, server: 127.0.0.1/127.0.0.1:2181
16:50:08,558 DEBUG zookeeper.ClientCnxn: Session establishment request sent on 127.0.0.1/127.0.0.1:2181
16:50:08,592  INFO zookeeper.ClientCnxn: Session establishment complete on server 127.0.0.1/127.0.0.1:2181, sessionid = 0x160ff8bf98b0007, negotiated timeout = 40000
16:50:08,592  INFO curator.framework.state.ConnectionStateManager: State change: CONNECTED
16:50:08,594 DEBUG zookeeper.ClientCnxn: Reading reply sessionid:0x160ff8bf98b0007, packet:: clientPath:null serverPath:null finished:false header:: 1,12  replyHeader:: 1,1345,0  request:: '/hiveserver2,F  response:: v{'serverUri=sandbox.hortonworks.com:10000;version=1.2.1000.2.5.0.0-1245;sequence=0000000007},s{467,467,1477381899524,1477381899524,0,15,0,0,10,1,1340} 
16:50:08,594 DEBUG curator.framework.imps.CuratorFrameworkImpl: Closing
16:50:08,594 DEBUG curator.CuratorZookeeperClient: Closing
16:50:08,594 DEBUG curator.ConnectionState: Closing
16:50:08,594 DEBUG zookeeper.ZooKeeper: Closing session: 0x160ff8bf98b0007
16:50:08,594 DEBUG zookeeper.ClientCnxn: Closing client for session: 0x160ff8bf98b0007
16:50:08,605 DEBUG zookeeper.ClientCnxn: Reading reply sessionid:0x160ff8bf98b0007, packet:: clientPath:null serverPath:null finished:false header:: 2,-11  replyHeader:: 2,1346,0  request:: null response:: null
16:50:08,605 DEBUG zookeeper.ClientCnxn: Disconnecting client for session: 0x160ff8bf98b0007
16:50:08,605  INFO zookeeper.ZooKeeper: Session: 0x160ff8bf98b0007 closed
16:50:08,605  INFO zookeeper.ClientCnxn: EventThread shut down
16:50:08,605 ERROR hive.jdbc.Utils: Unable to read HiveServer2 configs from ZooKeeper
java.sql.SQLException: Could not open client transport for any of the Server URI's in ZooKeeper: java.net.SocketTimeoutException: Read timed out
    at org.apache.hive.jdbc.HiveConnection.openTransport(HiveConnection.java:228)
    at org.apache.hive.jdbc.HiveConnection.<init>(HiveConnection.java:166)
    at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:105)
    at java.sql/java.sql.DriverManager.getConnection(DriverManager.java:678)
    at java.sql/java.sql.DriverManager.getConnection(DriverManager.java:252)
    at com.mycompany.hadoop.hiveconnect.Main.main(Main.java:13)
Caused by: org.apache.thrift.transport.TTransportException: java.net.SocketTimeoutException: Read timed out
    at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:129)
    at org.apache.thrift.transport.TTransport.readAll(TTransport.java:86)
    at org.apache.thrift.transport.TSaslTransport.receiveSaslMessage(TSaslTransport.java:178)
    at org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:307)
    at org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:37)
    at org.apache.hive.jdbc.HiveConnection.openTransport(HiveConnection.java:204)
    ... 5 more
Caused by: java.net.SocketTimeoutException: Read timed out
    at java.base/java.net.SocketInputStream.socketRead0(Native Method)
    at java.base/java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
    at java.base/java.net.SocketInputStream.read(SocketInputStream.java:171)
    at java.base/java.net.SocketInputStream.read(SocketInputStream.java:141)
    at java.base/java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
    at java.base/java.io.BufferedInputStream.read1(BufferedInputStream.java:286)
    at java.base/java.io.BufferedInputStream.read(BufferedInputStream.java:345)
    at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:127)
    ... 10 more

1 個解決方案

解決方案1
 0  2018-03-07 23:14:31

好吧，我無法解決這個問題。

但是后來，我下載了2.6.4沙箱，魔術般地發揮了作用！

暫無
暫無

聲明:本站的技術帖子網頁，遵循CC BY-SA 4.0協議，如果您需要轉載，請注明本站網址或者原文地址。任何問題請咨詢:yoyou2525@163.com.

相關問題 Hortonworks HDP Sandbox上的HBase：無法從ZooKeeper獲取主地址 使用HDP 2.2.4.2的Hortonworks Sandbox中的錯誤 無法將 Spark 與來自 eclipse 的 Hortonworks Sandbox 連接 從Hortonworks沙箱運行時出現配置單元錯誤 帶有 VirtualBox 的 Mac 上的 Hortonworks Sandbox HDP 2.6.5 hortonworks沙箱配置單元元存儲不起作用 Hive在HDP沙箱上添加JAR 如何使用HDP 2.4在Hortonworks沙箱中檢查Mapreduce作業日志 Hafonworks HDP 2.3 Sandbox中的Kafka控制台生產者錯誤 HDP2.5 Hortonworks Sandbox 忘記並重置 root 用戶密碼
相關標簽
 
粵ICP備18138465號  © 2020-2024 STACKOOM.COM