Hadoop 2.x — how to configure secondary namenode?

Question

I have an old Hadoop install that I'm looking to update to Hadoop 2. In the old setup, I have a $HADOOP_HOME/conf/masters file that specifies the secondary namenode.

Looking through the Hadoop 2 documentation I can't find any mention of a "masters" file, or how to setup a secondary namenode.

Any help in the right direction would be appreciated.

Answer 1

The slaves and masters files in the conf folder are only used by some scripts in the bin folder like start-mapred.sh, start-dfs.sh and start-all.sh scripts.

These scripts are a mere convenience so that you can run them from a single node to ssh into each master / slave node and start the desired hadoop service daemons.

You only need these files on the name node machine if you intend to launch your cluster from this single node (using password-less ssh).

Alternatively, You can also start an Hadoop daemon manually on a machine via

bin/hadoop-daemon.sh start [namenode | secondarynamenode | datanode | jobtracker | tasktracker]

In order to run the secondary name node, use the above script on the designated machines providing the 'secondarynamenode' value to the script

Answer 2

See @pwnz0r 's 2nd comment on answer on How separate hadoop secondary namenode from primary namenode?

To reiterate here:

In hdfs-site.xml:
<property> 
   <name>dfs.secondary.http.address</name>
   <value>$secondarynamenode.full.hostname:50090</value>
   <description>SecondaryNameNodeHostname</description>
</property>

Answer 3

I am using Hadoop 2.6 and had to use

<property> 
    <name>dfs.secondary.http.address</name>
    <value>secondarynamenode.hostname:50090</value>
</property>

for further details refer https://hadoop.apache.org/docs/r2.6.0/hadoop-project-dist/hadoop-hdfs/hdfs-default.xml

Answer 4

Update hdfs-site.xml file by updating and adding following property

cd  $HADOOP_HOME/etc/hadoop
sudo vi hdfs-site.xml

Then paste these lines into configuration tag

<property> 
    <name>dfs.secondary.http.address</name>
    <value>hostname:50090</value>
</property>