SLURM 批处理数组循环？

Question

I'm somewhat bash challenged and trying to send a large job array through slurm on my institution's cluster. I am way over my limit (which appears to be 1000 jobs per job array) and am having to iteratively parse out the list into blocks of 1000, which is tedious:

sbatch --array=17001-18000 -p <server-name> --time=12:00:00 <my-bash-script>

How might I write a loop to do this? Each job takes about 11 minutes, so I would need to build in a pause in the loop. Otherwise, I suspect SLURM will reject the new batch job. Anyone out there know what to do? Thanks in advance!

Answer 1

Something like this should do what you want

START=1
END=10000
STEP=1000
SLEEP=700 #Just over 11 Minutes (in seconds)

for i in $(seq $START $STEP $END) ; do
    JSTART=$i
    JEND=$[ $JSTART + $STEP - 1 ] 
    echo "Submitting with ${JSTART} and ${JEND}"
    sbatch --array=${JSTART}-${JEND} -p <server-name> --time=12:00:00 <my-bash-script>
    sleep $SLEEP
done