使用Livy连接到Kerberized Hadoop集群时，Python请求Post请求失败

Question

I'm trying to connect to a kerberized hadoop cluster via Livy to execute Spark code. The requests call im making is as below.

kerberos_auth = HTTPKerberosAuth(mutual_authentication=REQUIRED, force_preemptive=True)
r = requests.post(host + '/sessions', data=json.dumps(data), headers=headers, auth=kerberos_auth)

This call fails with the following error

GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos credentails)

Any help here would be appreciated.

Answer 1

When running Hadoop service daemons in Hadoop in secure mode, Kerberos tickets are decrypted with a keytab and the service uses the keytab to determine the credentials of the user coming into the cluster. Without a keytab in place with the right service principal inside of it, you will get this error message. Please refer to Hadoop in Secure Mode for further details on setting up the keytab.