Hive JDBC连接问题

Question

I am trying to connect to Hive2 server via JDBC with kerberos authentication. After numerous attempts to make it work, I can't get it to work with the Cloudera driver.

If someone can help me to solve the problem, I can greatly appreciate it.

I have this method:

    private Connection establishConnection() {
    final String driverPropertyClassName = "driver";
    final String urlProperty = "url";
    Properties hiveProperties = config.getMatchingProperties("hive.jdbc");
    String driverClassName = (String) hiveProperties.remove(driverPropertyClassName);
    String url = (String) hiveProperties.remove(urlProperty);
    Configuration hadoopConfig = new Configuration();
    hadoopConfig.set("hadoop.security.authentication", "Kerberos");
    String p = config.getProperty("hadoop.core.site.path");
    Path path = new Path(p);
    hadoopConfig.addResource(path);
    UserGroupInformation.setConfiguration(hadoopConfig);

    Connection conn = null;
    if (driverClassName != null) {
        try {
            UserGroupInformation.loginUserFromKeytab(config.getProperty("login.user"), config.getProperty("keytab.file"));
            Driver driver = (Driver) Class.forName(driverClassName).newInstance();
            DriverManager.registerDriver(driver);
            conn = DriverManager.getConnection(url, hiveProperties);
        } catch (Throwable e) {
            LOG.error("Failed to establish Hive connection", e);
        }
    }
    return conn;
}

URL for the server, that I am getting from the properties in the format described in Cloudera documentation

I am getting an exception:

2018-05-05 18:26:49 ERROR HiveReader:147 - Failed to establish Hive connection
java.sql.SQLException: [Cloudera][HiveJDBCDriver](500164) Error initialized or created transport for authentication: Peer indicated failure: Unsupported mechanism type PLAIN.
    at com.cloudera.hiveserver2.hivecommon.api.HiveServer2ClientFactory.createTransport(Unknown Source)
    at com.cloudera.hiveserver2.hivecommon.api.ZooKeeperEnabledExtendedHS2Factory.createClient(Unknown Source)
...

I thought, that it is missing AuthMech attribute and added AuthMech=1 to the URL. Now I am getting:

java.sql.SQLNonTransientConnectionException: [Cloudera][JDBC](10100) Connection Refused: [Cloudera][JDBC](11640) Required Connection Key(s): KrbHostFQDN, KrbServiceName; [Cloudera][JDBC](11480) Optional Connection Key(s): AsyncExecPollInterval, AutomaticColumnRename, CatalogSchemaSwitch, DecimalColumnScale, DefaultStringColumnLength, DelegationToken, DelegationUID, krbAuthType, KrbRealm, PreparedMetaLimitZero, RowsFetchedPerBlock, SocketTimeOut, ssl, StripCatalogName, transportMode, UseCustomTypeCoercionMap, UseNativeQuery, zk
    at com.cloudera.hiveserver2.exceptions.ExceptionConverter.toSQLException(Unknown Source)
    at com.cloudera.hiveserver2.jdbc.common.BaseConnectionFactory.checkResponseMap(Unknown Source)
    ...

But KrbHostFQDN is already specified in the principal property as required in the documentation.

Am I missing something or is this documentation wrong?

Answer 1

Below is the one of the similar kind of problem statement in Impala (just JDBC engine changes others are same) that is resolved by setting "KrbHostFQDN" related properties in JDBC connection string itself.

Try to use the URL below. Hopefully works for u.

String jdbcConnStr = "jdbc:impala://myserver.mycompany.corp:21050/default;SSL=1;AuthMech=1;KrbHostFQDN=myserver.mycompany.corp;KrbRealm=MYCOMPANY.CORP;KrbServiceName=impala"

I suppose that if you are not using SSL=1 but only Kerberos, you just drop that part from the connection string and don't worry about setting up SSL certificates in the java key store, which is yet another hassle.

However in order to get Kerberos to work properly we did the following:

• Install MIT Kerberos 4.0.1, which is a kerberos ticket manager. (This is for Windows)

• This ticket manager asks you for authentication every time you initiate a connection, creates a ticket and stores it in a kerberos_ticket.dat binary file, whose location can be configured somehow but I do not recall exactly how.

• Finally, before launching your JAVA app you have to set an environment variable KRB5CCNAME=C:/path/to/kerberos_ticket.dat . In your java app, you can check that the variable was correctly set by doing System.out.println( "KRB5CCNAME = " + System.getenv( "KRB5CCNAME" ) ) . If you are working with eclipse or other IDE you might even have to close the IDE,set up the environment variable and start the IDE again.

  • NOTE: this last bit is very important, I have observed that if this variable is not properly set up, the connection wont be established...

• In Linux, instead MIT Kerberos 4.0.1, there is a program called kinit which does the same thing, although without a graphical interface, which is even more convenient for automation.

Answer 2

I wanted to put it in the comment but it was too long for the comment, therefore I am placing it here: I tried your suggestion and got another exception:

java.sql.SQLException: [Cloudera]HiveJDBCDriver Error creating login context using ticket cache: Unable to obtain Principal Name for authentication .

May be my problem is, that I do not have environment variable KRB5CCNAME set. I, honestly, never heard about it before. What is supposed to be in that ticket file. I do have, however, following line in my main method:

System.setProperty("java.security.krb5.conf", "path/to/krb5.conf");

Which is supposed to be used by

UserGroupInformation.loginUserFromKeytab(config.getProperty("login.user"), config.getProperty("keytab.file"));

to obtain the kerberos ticket.

Answer 3

To solve this issue update Java Cryptography Extension for the Java version that you use in your system.

1. Here's the link when you can download JCE for Java 1.7
2. Uncompress and overwrite those files in $JDK_HOME/jre/lib/security
3. Restart your computer.