简体繁体 English

多节点hadoop集群安装

[英]Multi-node hadoop cluster installation

原文 2015-08-17 22:47:37 6 2 hadoop/ bigdata/ cloudera

Sorry if my question appears to be naïve. 抱歉，如果我的问题看起来很幼稚。 We are planning to use CDH 5.3.0 or 5.4.0. 我们计划使用CDH 5.3.0或5.4.0。 We want to implement a multi-node cluster. 我们要实现一个多节点集群。

The example multi-node installations that I have seen/read on different blogs/resources have master and slaves on different hosts. 我在不同的博客/资源上看到/阅读的示例多节点安装在不同的主机上具有主服务器和从服务器。

However, we are restrained by the number of hosts. 但是，我们受到主机数量的限制。 We have only 2 powerful hosts ( 32 cores 400+ GB RAM), so if we decide to have master on one and slave on other, we will end up with only one slave. 我们只有2个功能强大的主机（32核400+ GB RAM），因此，如果我们决定一个主机为主，另一主机为从机，那么最终将只有一个从机。 My questions are : 我的问题是：

Is it possible to have master and slave on the same hosts? 主服务器和从服务器可以在同一主机上吗？
Can I have more than one slave node on a single host. 一台主机上可以有多个从属节点吗？
Also does one need to pay to use Cloudera Manager or it is open-source like the rest of the components. 使用Cloudera Manager还是需要付费的，或者像其他组件一样是开源的。

If you can point me in the direction of some resource which would help me understand above scenarios it would be helpful. 如果您可以向我指出一些资源的方向，这将有助于我理解上述情况，那么它将很有帮助。

Thanks for your help. 谢谢你的帮助。

Regards, 问候，

V V

2 个解决方案

old question but no and wrong answer: 旧问题，但没有错答案：

yes, it is possible to install Master & Worker services on a single host. 是的，可以在单个主机上安装Master＆Worker服务。 eg HDFS (NameNode and Datanode). 例如HDFS（NameNode和Datanode）。 You can even install a full cloudera or Hortonworks installation with ALL services on a single host if it is powerfull enough, but i would only recommend it for POC or testcases. 如果功能足够强大，您甚至可以在单个主机上安装带有所有服务的完整cloudera或Hortonworks安装，但是我只建议将其用于POC或测试用例。
If you use cloudera or hortonworks without virtualization it is not possible to run multiple instances of the SAME worker services eg datanode on the same host. 如果您在没有虚拟化的情况下使用cloudera或hortonworks，则无法在同一主机上运行SAME辅助服务的多个实例，例如datanode。 1 Host 1 worker instance. 1个主机1个工作实例。 everything else would not make sense. 其他一切都没有意义。
Cloudera is a package of multiple open source projekt (Hadoop,Spark....) and other closed source parts like cloudera manager and other enterprise closed source features. Cloudera是一个包含多个开放源代码项目（Hadoop，Spark ....）和其他封闭源代码部分（如cloudera manager和其他企业封闭源功能）的程序包。 But everything you need is free even for commercial use with the community licence. 但是，即使您获得社区许可，也可以免费用于商业用途。 Right now (2017): only cloudera navigator is the big feature which is not part of the community edition 目前（2017）：只有cloudera导航器才是主要功能，不属于社区版

Yes you can configure namenode and datanode both on a single node. 是的，您可以在单个节点上同时配置namenode和datanode。
You cannot have more than two datanodes on a single machine. 一台机器上不能有两个以上的数据节点。
Cloudera is open-source hadoop distribution. Cloudera是开源的hadoop发行版。