CDH4 : Add new node to existing cluster

Question

I have successfully created hadoop cluster with CDH4 on ubuntu . I have created this with one master(master) and one slave(slave1) . Now I want to add one more cluster . For this I just cloned slave2 and updated hosts and ssh accordingly . Then I updated conf/slaves file with all datanode dns names in all nodes and restarted everything . But it's not detecting the new datanode instead it only shows the old one that is slave1 not slave2 . Can anyone please help me on this ? I have used cdh4-repository_1.0_all.deb

Answer 1

@user2009755, you need to create a master and slave file only in the master. And in configuration files in $HADOOP_HOME/etc/hadoop, make necessary changes to the URI pointing to the master node.

NOTE: Try to format the namenode and delete the tmp files (usually /tmp/*) but if you changed it in core-site.xml , format that directory in all nodes and start all the daemons, it worked for me.

Answer 2

There is so many reasons,

Have you change the dfs.replication value to 3 in conf/hdfs-site.xml??

check on master with cammands hduser@master:~$ ssh slave it should be show the slave terminal if not then execute this cammand - hduser@master:~$ ssh-copy-id -i $HOME/.ssh/id_rsa.pub hduser@slave

for fully understand see this link

http://www.michael-noll.com/tutorials/running-hadoop-on-ubuntu-linux-multi-node-cluster/