繁体   English   中英

R和雪在亚马逊EC2使用starcluster

[英]R and snow on amazon EC2 using starcluster

我正在尝试在AWS EC2集群上以R并行运行分析。 我正在使用starcluster来设置和管理EC2集群,并尝试在R中使用snowforeach 。首先,我在集群中有2个节点,1个master和1个worker。

starcluster start mycluster 
starcluster listinstances

-----------------------------------------
mycluster (security group: @sc-mycluster)
-----------------------------------------
....
Cluster nodes:
     master running i-xxxxxxxxx masterIP.compute-1.amazonaws.com
    node001 running i-xxxxxxxxx node001IP.compute-1.amazonaws.com
Total nodes: 2

starcluster sshmaster mycluster

然后我启动R并加载snow包并尝试创建一个集群对象。

R
library("snow")

cl = makeCluster(c("masterIP.compute-1.amazonaws.com", "node001IP.compute-1.amazonaws.com"), type = "SOCK")

但是,这会给我以下错误消息:

The authenticity of host 'masterIP.compute-1.amazonaws.com (xx.xxx.xx.xx)' can't be established.
ECDSA key fingerprint is xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'masterIP.compute-1.amazonaws.com,xx.xxx.xx.xx' (ECDSA) to the list of known hosts.
Permission denied (publickey).

所以我尝试将我的ssh密钥( keyname.rsa具体)复制到EC2上的.ssh文件中并再次尝试。 那还是行不通; 我收到了相同的Permission denied (publickey). 错误。 我认为starcluster处理节点之间的ssh和通信的设置,所以我有点困惑为什么我无法设置它。 我也尝试添加node001,所以cl = makeCluster(c("node001IP.compute-1.amazonaws.com"), type = "SOCK") ,但是发生了同样的错误。

事实证明,经过多次修补后,所需要的只是R版本2.15的更新。 命令cl = makeCluster(c("masterIP.compute-1.amazonaws.com", "node001IP.compute-1.amazonaws.com"), type = "SOCK")之后完美运行。

暂无
暂无

声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM