简体繁体 English

并行执行多个作业时指定 SLURM 资源

[英]Specifying SLURM Resources When Executing Multiple Jobs in Parallel

原文 2018-12-28 00:48:41 3 1 bash/ slurm/ sbatch

According to the answers here What does the --ntasks or -n tasks does in SLURM?根据这里的答案，--ntasks 或 -n tasks 在 SLURM 中做了什么？ one can run multiple jobs in parallel via ntasks parameter for sbatch followed by srun .可以通过ntasks参数并行运行多个作业，然后是srun 。 To ask a follow up question - how would one specify the amount of memory needed when running jobs in parallel like so?问一个后续问题 - 像这样并行运行作业时，如何指定所需的内存量？

If say 3 jobs are running in parallel each needing 8G of memory, would one specify 24G of memory in sbatch (ie the sum of memory from all jobs) or not give memory parameters in sbatch but instead specify 8G of memory for each srun ?如果说 3 个作业并行运行，每个都需要 8G 内存，是否会在sbatch指定 24G 内存（即所有作业的内存总和），或者不在sbatch提供内存参数而是为每个srun指定 8G 内存？

1 个解决方案

You need to specify the memory requirement in the script submitted with sbatch , otherwise you will end up with the default memory allocation, which might not correspond to your needs.您需要在使用sbatch提交的脚本中指定内存要求，否则您最终会得到默认的内存分配，这可能不符合您的需求。 If you then specify the 8GB memory in the srun call, you might end up with no jobs being able to start if the default memory is lower than that, or having only one or two jobs running in parallel if the default memory is between 16 and 24GB.如果您随后在srun调用中指定 8GB 内存，如果默认内存低于该值，您可能最终无法启动任何作业，或者如果默认内存介于 16 和24GB。

You can request --mem=24GB , but that offer less flexibility than specifying --mem-per-cpu=8G .您可以请求--mem=24GB ，但这提供的灵活性不如指定--mem-per-cpu=8G 。