Authenticating to Kerberos enabled Hadoop Cluster Using Java

Question

I have Kerberos and have enabled Hadoop cluster. I need to perform HDFS operations using Java code.

I have keytab file and username.

Can someone please suggest how can I authenticate Kerberos using Java code?

Thanks.

Answer 1

If you just want to use the HDFS Java API for a short-lived connection, then UserGroupInformation has a loginUserFromKeytab method.

If you want a durable connection (ie longer than the lifetime of the initial Kerberos TGT) then have a look at that post .

If you are interested in the dark side of Kerberos (well, there is no other side anyway) then have a look at the "Madness beyond the Gate" online book -- at your own risk.

~~~~~~

BTW there is an alternative to the Java API: the WebHDFS REST service . But on Windows you still need some Java code to create the Kerberos TGT (using GSSAPI ie the Linux way, not SSPI ie the Microsoft way!!!) , run the SPNEGO initial authentication, and retrieve the delegation token. At this point any HTTPS library will be able to handle the file operations using that token.