在Slurm群集上运行批处理作业

Question

So I have spent a few hours now trying to figure this out and would appreciate any help.

What I am trying to do is run a batch job with a slurm --array0-654

I would like each job step to run 8 threads.

I have access to 11 nodes on the cluster each capable of 32 threads and I can't seem to get the SBATCH configuration correct. It seems to either run one job per node or run over 200 of the jobs across all nodes.

I have tried a number of different configs and was wondering if anyone has some suggestions.

Answer 1

From your description, you should use

--array 0-654
--ntasks 1
--cpus-per-task 8

but the outcome will depend on the cluster configuration (notably whether or not the nodes can be shared among nodes, presence of limits on the number of jobs, etc.)