在 SLURM 批处理脚本中将参数传递给 python 脚本

Question

I've written a python script that requires two arguments and works just fine when I run it on the command line with:

pythonscript.py arg1 arg2

I need to run this in a SLURM batch script, but whenever I do I get an "illegal instruction" error and a core dump. This is an example of the script I'm using:

#!/bin/bash
# Set your minimum acceptable walltime, format: day-hours:minutes:seconds
#SBATCH --time=0-00:30:00

# Set name of job shown in squeue
#SBATCH --job-name pythonscript

# Request CPU resources
#SBATCH --ntasks=1
#SBATCH --ntasks-per-node=1
#SBATCH --cpus-per-task=1

# Memory usage (MB)
#SBATCH --mem-per-cpu=3000

# Use modules to set the software environment
module purge
module load python/2.7.8-gcc

pythonscript.py arg1 arg2

I've spent a lot of time trying to figure out exactly what is causing the core dumps, and this is what I've narrowed it down to. It only crashes when run from the batch script and only when I'm trying to run the script with arguments. When I modify it to run without arguments, it runs properly. Can anyone tell me how to pass these arguments to my python script within a SLURM script?

Answer 1

This may seem like a silly question, but is the pythonscript.py executable?

Also have you tried

python pythonscript.py arg1 arg2

Alternately, you might be able to change

#!/bin/bash

to

#!/bin/python

or the appropriate path to the python version you want to use.

Answer 2

The problem was the node where my job was sent. I managed to check in which node my job run without errors with the command:

sacct --format=JobID,"AllocCPUS,Node,JobName%20,MaxVMSize,MaxRSS,ReqMem,Start,End"

I identified the proper node and then switch in my .sh with:

#SBATCH -w, --nodelist=compute124

Answer 3

I use the following and I can add any number of arguments.

#!/bin/bash
#SBATCH --cpus-per-task=1        # number of cores

cd $SLURM_SUBMIT_DIR
pwd

python $SLURM_SUBMIT_DIR/python.py HumptyDumpty.txt newhumpty.txt ad.txt