为什么在 slurm 集群上并行化 R 代码时只有一个节点工作

Question

My testing code is

rm(list = ls())

library(parallel)
#setwd("/home/121090562/TestWorkspace/")

fun <- function(x){
return (x+1);
}

cl <- makeCluster(80)
ptm <- proc.time()
system.time({
res <- parLapply(cl, 1:50000000,  fun)
});
print(proc.time() - ptm)
stopCluster(cl)

I want to use two nodes, each node has 40 CPUs, but only one node works instead of 2, which means only 40 CPUs work.

the job file is

#!/bin/bash

#SBATCH --job-name Cores.R
#SBATCH --partition warshel-cpu
#SBATCH --nodes=2
#SBATCH --time 1-00:00:00
#SBATCH --ntasks 80
#SBATCH --output /...
#SBATCH --error /...
cd /...
R --vanilla --slave < Cores.R

Answer 1

Without using something like the Rmpi package (or SlurmCluster (?)), R has no way of doing inter-node communication. Your current setup is limited to a single node only.