Google Compute Engine上的Hadoop

Question

I am trying to setup hadoop cluster in Google Compute Engine through "Launch click-to-deploy software" feature .I have created 1 master and 1 slave node and tried to start the cluster using start-all.sh script from master node and i got error "permission denied(publickey)" .

I have generated public and private keys in both slave and master nodes .

currently i logged into the master with my username, is it mandatory to login into master as "hadoop" user .If so ,what is the password for that userid .

please let me know how to overcome this problem .

Answer 1

The deployment creates a user hadoop which owns Hadoop-specific SSH keys which were generated dynamically at deployment time; this means since start-all.sh uses SSH under the hood, you must do the following:

sudo su hadoop
/home/hadoop/hadoop-install/bin/start-all.sh

Otherwise, your "normal" username doesn't have SSH keys properly set up so you won't be able to launch the Hadoop daemons, as you saw.

Another thing to note is that the deployment should have already started all the Hadoop daemons automatically, so you shouldn't need to manually run start-all.sh unless you're rebooting the daemons after some manual configuration updates. If the daemons weren't running after the deployment ran, you may have encountered some unexpected error during initialization.