Python slurm 集群上的作业，节点与核心

Question

I have an extremely basic question, to which I somehow never managed to find an answer. Let's assume that I have access to a cluster running slurm, and that I need to run a Python job on the cluster. Let us assume that my code has not been written to support multiprocessing. Do I have any reason to require multiple cores? Or should I then stick to 1 node and 1 core?

Conversely, if I wanted to run 5 times the same script (with different input variables for example), is there any difference between requiring 1 node and assigning 1 core to each job, or requiring 5 nodes with 1 core each?

Answer 1

If your python script does not use multiple threads, then yes, you should stick to one task ( -n1 ) with one CPU ( -c1 ) on one node ( -N1 ). You don't need to specify that as it is the default anyway. If you request more resources, they will just be wasted, as you don't use them.

However: Some python libraries to multithreaded calculations without the need to specify them explicitly, so if you do some numpy calculations, you may benefit from multiple cores .

Conversely, if I wanted to run 5 times the same script (with different input variables for example), is there any difference between requiring 1 node and assigning 1 core to each job, or requiring 5 nodes with 1 core each?

Yes, there is: If you ask Slurm for 5 Nodes with 1 task per node, then it will have to wait until 5 nodes have room for a task. Even if for example a node with 20 CPUs is completely empty, your job won't run, as you explicitly asked for 5 nodes. So I would advise to start 5 jobs with -n1 .