使用MapDriver测试Hbase连接

Question

I'm using MapDriver to test the Mapper program. In the Mapper - setup() method I'm instantiating a Hbase table and it is trying to connect to the zookeeper.
In my test class I'm not able to connect to the zookeeper and it keeps failing with connection timeout exception.

SampleMapper.java
@Override
public void setup(Context context) throws IOException, InterruptedException {
    Configuration config = context.getConfiguration();
    table = new HTable(config, config.get("tableName"));
    Scan tableScan = new Scan();
    tableScan.addColumn(COLUMN_FAMILY_STUDENT, COLUMN_ID);
    tableScan.setCaching(NUMBER_OF_ROWS_TO_CACHE);
    tableScan.setCacheBlocks(false);
    super.setup(context);
}

sampleMapperTest.java
@Before
public void setUp() {
    mapDriver = MapDriver.newMapDriver(new SampleMapper());
    Configuration conf = mapDriver.getConfiguration();
    conf.set("tableName", "testTable");
    conf.setStrings("io.serializations",
            conf.get("io.serializations"), MutationSerialization.class.getName(), ResultSerialization.class.getName());

}


 [org.apache.zookeeper.ZooKeeper] Initiating client connection, connectString=localhost:2181 sessionTimeout=180000 watcher=hconnection-0x625e134e, quorum=localhost:2181, baseZNode=/hbase
  main-SendThread(127.0.0.1:2181) INFO  [org.apache.zookeeper.ClientCnxn] Opening socket connection to server 127.0.0.1/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error)
  main-SendThread(127.0.0.1:2181) WARN  [org.apache.zookeeper.ClientCnxn] Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
    java.net.ConnectException: Connection refused: no further information
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
        04 Mar 2018 20:31 47,648 main-SendThread(0:0:0:0:0:0:0:1:2181) INFO  [org.apache.zookeeper.ClientCnxn] Opening socket connection to server 0:0:0:0:0:0:0:1/0:0:0:0:0:0:0:1:2181. Will not attempt to authenticate using SASL (unknown error)
        04 Mar 2018 20:31 47,702 main WARN  [org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper] Possibly transient ZooKeeper, quorum=localhost:2181, exception=org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/hbaseid
         04 Mar 2018 20:31 47,702 main INFO  [org.apache.hadoop.hbase.util.RetryCounter] Sleeping 1000ms before retry #0...
        04 Mar 2018 20:31 48,651 main-SendThread(0:0:0:0:0:0:0:1:2181) WARN  [org.apache.zookeeper.ClientCnxn] Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
        java.net.ConnectException: Connection refused: no further information
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
            04 Mar 2018 20:31 48,752 main WARN  [org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper] Possibly transient ZooKeeper, quorum=localhost:2181, exception=org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/hbaseid
       04 Mar 2018 20:31 48,752 main INFO  [org.apache.hadoop.hbase.util.RetryCounter] Sleeping 2000ms before retry #1...
       04 Mar 2018 20:31 49,753 main-SendThread(127.0.0.1:2181) INFO  [org.apache.zookeeper.ClientCnxn] Opening socket connection to server 127.0.0.1/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error)
       04 Mar 2018 20:31 50,755 main-SendThread(127.0.0.1:2181) WARN  [org.apache.zookeeper.ClientCnxn] Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
    java.net.ConnectException: Connection refused: no further information

I'm not sure how to mock the zooker here. How can i get the connection to zookeeper?

Can anyone help on this?

Answer 1

main-SendThread(127.0.0.1:2181) INFO [org.apache.zookeeper.ClientCnxn] Opening socket connection to server 127.0.0.1/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error) main-SendThread(127.0.0.1:2181) WARN [org.apache.zookeeper.ClientCnxn] Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused: no further information

Your error shows that you are connecting to localhost (127.0.0.1). If you are using remote host then set following:

config.set("hbase.zookeeper.quorum", "remote ip");
config.set("hbase.zookeeper.property.clientPort", "2181"); //your port
config.set("hbase.master", "hbase_master_ip:16000"); //your port