Hive 和 Hdinsight 与 Azure 上的 python 的连接

Question

I want to connect the hive with hdinsight cluster with python script, but due to access issue not able to access cluster dashboard which will give me JDBC url information which helps in making the connection with hive.

Can anyone please help/recommend me the ways to connect hive with hdinsight cluster with help of python.

import jaydebeapi
conn = jaydebeapi.connect("org.apache.hive.jdbc.HiveDriver",
       "jdbc:hive2://16.4.5.492:451/default/;ssl=true;transportMode=http;httpPath=/hive2", 
       ['Username', 'Password'],
       ["/jdbc/hive-jdbc-1.2.1-standalone.jar",
        "/jdbc/httpclient-4.4.jar",
        "/jdbc/httpcore-4.4.4.jar"])

Answer 1

WebHCat is a REST API for HCatalog, a table, and storage management layer for Apache Hadoop. WebHCat is enabled by default on HDInsight clusters , and is used by various tools to submit jobs, get job status, etc. without logging in to the cluster. So in a nutshell you can use WebHCat to run the Hive QL in Python.

Additional reference:

https://cwiki.apache.org/confluence/display/Hive/WebHCat https://social.msdn.microsoft.com/Forums/en-US/b8026143-975f-46c9-b937-99f604e3b757/connect-to-hive-in-hdinsight-using-python-app-on-remote-linuxbased-server?forum=hdinsight

Additionally you can refer below documentation for making it work with JDBC driver:

https://github.com/uglide/azure-content/blob/master/articles/hdinsight/hdinsight-connect-hive-jdbc-driver.md

I haven't tried this but once the configuration is completed, you should be able to use below code for querying.

import jaydebeapi
conn = jaydebeapi.connect("org.apache.hive.jdbc.HiveDriver",
       "jdbc:hive2://my_ip_or_url:443/;ssl=true;transportMode=http;httpPath=/hive2", 
       [username, password],
       "/jdbc/hive-jdbc-1.2.1.jar")

Hope it helps.