My testing code is
rm(list = ls())
library(parallel)
#setwd("/home/121090562/TestWorkspace/")
fun <- function(x){
return (x+1);
}
cl <- makeCluster(80)
ptm <- proc.time()
system.time({
res <- parLapply(cl, 1:50000000, fun)
});
print(proc.time() - ptm)
stopCluster(cl)
I want to use two nodes, each node has 40 CPUs, but only one node works instead of 2, which means only 40 CPUs work.
the job file is
#!/bin/bash
#SBATCH --job-name Cores.R
#SBATCH --partition warshel-cpu
#SBATCH --nodes=2
#SBATCH --time 1-00:00:00
#SBATCH --ntasks 80
#SBATCH --output /...
#SBATCH --error /...
cd /...
R --vanilla --slave < Cores.R
Without using something like the Rmpi
package (or SlurmCluster
(?)), R has no way of doing inter-node communication. Your current setup is limited to a single node only.
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.