[英]R and snow on amazon EC2 using starcluster
我正在尝试在AWS EC2集群上以R并行运行分析。 我正在使用starcluster来设置和管理EC2集群,并尝试在R中使用snow
和foreach
。首先,我在集群中有2个节点,1个master和1个worker。
starcluster start mycluster
starcluster listinstances
-----------------------------------------
mycluster (security group: @sc-mycluster)
-----------------------------------------
....
Cluster nodes:
master running i-xxxxxxxxx masterIP.compute-1.amazonaws.com
node001 running i-xxxxxxxxx node001IP.compute-1.amazonaws.com
Total nodes: 2
starcluster sshmaster mycluster
然后我启动R并加载snow
包并尝试创建一个集群对象。
R
library("snow")
cl = makeCluster(c("masterIP.compute-1.amazonaws.com", "node001IP.compute-1.amazonaws.com"), type = "SOCK")
但是,这会给我以下错误消息:
The authenticity of host 'masterIP.compute-1.amazonaws.com (xx.xxx.xx.xx)' can't be established.
ECDSA key fingerprint is xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'masterIP.compute-1.amazonaws.com,xx.xxx.xx.xx' (ECDSA) to the list of known hosts.
Permission denied (publickey).
所以我尝试将我的ssh密钥( keyname.rsa
具体)复制到EC2上的.ssh文件中并再次尝试。 那还是行不通; 我收到了相同的Permission denied (publickey).
错误。 我认为starcluster处理节点之间的ssh和通信的设置,所以我有点困惑为什么我无法设置它。 我也尝试添加node001,所以cl = makeCluster(c("node001IP.compute-1.amazonaws.com"), type = "SOCK")
,但是发生了同样的错误。
事实证明,经过多次修补后,所需要的只是R版本2.15的更新。 命令cl = makeCluster(c("masterIP.compute-1.amazonaws.com", "node001IP.compute-1.amazonaws.com"), type = "SOCK")
之后完美运行。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.