尝试通过在python中使用pyhs2连接到配置单元| 错误：ModuleNotFoundError：没有名为“ cloudera”的模块

Question

I am trying to connect to Hive from windows locally using python : below is the code that i used :

import pyhs2 as hive
DEFAULT_DB = 'default'
DEFAULT_SERVER = '10.37.40.1'
DEFAULT_PORT = 10000
DEFAULT_DOMAIN = 'PAM01-PRD01.IBM.COM'
u = "username"
s = "password"
# Build the Hive Connection
connection = hive.connect(host=DEFAULT_SERVER, port= DEFAULT_PORT, user=u + '@' + DEFAULT_DOMAIN, password=s)
# Hive query statement
statement = "select * from user_yuti.Temp_CredCard where pir_post_dt = '2014-05-01' limit 100"
cur = connection.cursor()

# Runs a Hive query and returns the result as a list of list
cur.execute(statement)
df = cur.fetchall() 

and execute that script via cmd :

python hive-connection-test.py

But I am getting error -

Do guys have any idea that has happened here!! they is not package called cloudera in python package library, i have search for it.

Answer 1

Sorry, but I'm only going to half solve this questions as I'm only halfway through fixing it myself...

connections.py (located in Lib/site-packages/pyhs2/)has an import:

from cloudera.thrift_sasl import TSaslClientTransport

This is trying to import the thrift_sasl from the clouders subdirectory, so the syntax should actually be:

from .cloudera.thrift_sasl import TSaslClientTransport

You can go in and change it yourself, but then it turns out that thrift_sasl.py is trying cStringIO which is no longer available in Python 3.0... so trying to find a workaround for this now.

Hope this helps you along.

Answer 2

The error was caused by the import statement used in pyhs2. The import feature was supported by Python2 but not Python3. You can have a try using Python2.7 instead of Python3 (you are currently using).

pyhs2 is no longer maintained for years, so it's better to use other alternatives such as PyHive which is supported by Dropbox official.