Hadoop名称节点不侦听外部IP

Question

I have a simple hadoop cluster of 3 machines. One master and two slaves. All of them are running the latest minimalist version of CentOS I downloaded there .

All my machines have the same hosts file:

127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.56.101  master
192.168.56.102  slave1
192.168.56.103  slave2

The core-site.xml file is configured as follows: (The namenode should listen on port 9000)

<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>

<!-- Put site-specific property overrides in this file. -->

<configuration>
<property>
    <name>fs.default.name</name>
    <value>hdfs://master:9000/</value>
</property>
<property>
    <name>dfs.permissions</name>
    <value>false</value>
</property>
</configuration>

When starting hdfs from the master and checking the logs of the datanodes/slaves I noticed the datanodes could not connect to the namenode .

Retrying connect to server: master/192.168.56.101:9000. Already tried 0 time(s).

After further checking it looks like the port 9000 is only accessible if I attempt to connect to it from the namenode itself.

[root@HadoopMaster conf]# sudo netstat -tulpn | grep :9000
tcp6       0      0 192.168.56.101:9000     :::*                    LISTEN      11055/java

Attempting a telnet master 9000 only succeeds from the master host. The slaves get the following error:

[root@Slave1 logs]# telnet master 9000
Trying 192.168.56.101...
telnet: connect to address 192.168.56.101: No route to host

But on the contrary they can connect to port 22 just fine

[root@Slave1 logs]# telnet master 22
Trying 192.168.56.101...
Connected to master.
Escape character is '^]'.
SSH-2.0-OpenSSH_6.6.1

So this has to be an IP binding issue . I have looked everywhere and could not find a viable answer. Similar questions to mine were solved by using a fully qualified domain name in the core-site.xml file which I did without success.

By the way, I also disabled iptables and before that opened all my master's ports to my slaves IPs. No success.

If you find the answer, I would definitely like to send you a delicious virtual cookie ;)

Answer 1

I just realized CentOS's default firewall is not iptables but firewalld. After configuring this firewall correctly I managed to solve my problem!

Why is iptables installed but not used, that is a mystery though. Boy, I prefer so much the Debian family.

Answer 2

If you are running those VM in several instances of VM VirtualBox that seems natural, even if they are on the same network they can't see each other.

Check virtualbox manual to configure the network properly, you can also check this answer