How to authenticate to Hadoop from command line? Removing `ls: SIMPLE authentication is not enabled` error

Question

I am setting up Kerberos authentication on a Hadoop cluster. From a machine outside the cluster, whenever I do hadoop fs -ls , I get the following message: ls: SIMPLE authentication is not enabled. Available:[TOKEN, KERBEROS] ls: SIMPLE authentication is not enabled. Available:[TOKEN, KERBEROS] , which is nice, since that is what I want. But now my question is, how do I actually execute that command succesfully? I know that I cannot proceed if I don't authenticate, but how do I authenticate from the command line?

Answer 1

Try below steps

Server

kadmin.local
addprinc user@realm.com

Client

kinit user@realm.com

klist to view the principals

Answer 2

Things need to be modified both in the Hadoop server and in the client that is trying to access it.

Server

In the server you need to add a new user to the realm, and for that you use the following commands:

In the kadmin.local shell, type the following command: addprinc user@realm.com . When prompted, type the password twice.

Client

Be sure that you have kinit installed and pointing to the Kerberos server of the cluster, and run the following command to acquire a token from the Kerberos server: kinit user@realm.com . To verify that you succeeded acquiring it, type klist .

And last, but not least: be sure that your client core-site.xml file is in agreement with the corresponding server-side configuration. Particularly, be sure that the value for the key hadoop.security.authentication is not simple in your client if it is not simple in your server.