YARN集群模式下，如何设置主节点memory大于工作节点memory？
[英]In YARN cluster mode, how to set memory the master node more than workers nodes' memory?
问题描述
In my algorithm, master node needs more memory (say 20GB) while the worker nodes need much less memory (say 3GB).
在我的算法中，主节点需要更多的 memory（比如 20GB），而工作节点需要更少的 memory（比如 3GB）。 However, as far as I know, in H2O it is only possibly to set the master node the same memory as worker nodes using -mapperXmx .但是，据我所知，在 H2O 中，只能使用 -mapperXmx 将主节点设置为与工作节点相同的-mapperXmx 。 Is it possible to set the master node 20GB of memory and workers nodes' memory 3GB each?是否可以将主节点的 memory 和工作节点的 memory 各 3GB 设置为 20GB？
1 个解决方案
解决方案1
0 2021-12-01 15:01:08
The master node is also known as the "driver" so yes, you can set the driver memory: spark.driver.memory Here's a complete list of settings you can tweak.主节点也称为“驱动程序”，所以是的，您可以设置驱动程序 memory： spark.driver.memory这是您可以调整的完整设置列表。
声明:本站的技术帖子网页，遵循CC BY-SA 4.0协议，如果您需要转载，请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.