简体   繁体   English

单节点集群中的 h2o 是进行并行处理还是仅在多节点集群中进行并行处理?

[英]Does h2o in a single node cluster do parallel processing or is it only in multi node cluster that parallel processing kicks in?

We are running h2o as a single node cluster inside in AWS:我们在 AWS 中将 h2o 作为单节点集群运行:

R is connected to the H2O cluster: 
    H2O cluster uptime:         5 seconds 217 milliseconds 
    H2O cluster timezone:       Etc/UTC 
    H2O data parsing timezone:  UTC 
    H2O cluster version:        3.17.0.4153 
    H2O cluster version age:    10 months and 4 days !!! 
    H2O cluster name:           h2o-8ba55ebb-7d49-41bd-b4e2-d7be45b5f53e 
    H2O cluster total nodes:    1 
    H2O cluster total memory:   22.20 GB 
    H2O cluster total cores:    8 
    H2O cluster allowed cores:  8 
    H2O cluster healthy:        TRUE 
    H2O Connection ip:          localhost 
    H2O Connection port:        54321 
    H2O Connection proxy:       NA 
    H2O Internal Security:      FALSE 
    H2O API Extensions:         XGBoost, Algos, AutoML, Core V3, Core V4 
    R Version:                  R version 3.4.3 (2017-11-30) 

And starting h2o from java with nthreads -1:并使用 nthreads -1 从 java 启动 h2o:

java -ea -Xmx25g -jar /path/to/h2o.jar -name unique-cloud-name 
     -ip localhost -ice_root /tmp/h2o-tmp -nthreads -1

We're wondering if with a single node cluster that h2o is doing parallel processing / using all available and allowed cores.我们想知道 h2o 是否在单节点集群中进行并行处理/使用所有可用和允许的内核。 When we do top -H in the commandline we do see coincidentally 8 active java processes and wondering if those are from h2o and are helping generate our model.当我们在命令行中执行 top -H 时,我们确实看到了 8 个活动的 java 进程,并想知道它们是否来自 h2o 并帮助生成我们的模型。

在此处输入图片说明

Yes, H2O will use all the cores on a single node to train one model.是的,H2O 将使用单个节点上的所有内核来训练一个模型。

nthreads lets you explicitly set the thread pool size that controls the amount of parallelism per process. nthreads 允许您显式设置控制每个进程的并行量的线程池大小。

声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM