正确将命令输出转义为SLURM提交中的变量

Question

I have a master script that submits a script using standard SLURM submission like so:

                    cat > $jobfile <<EOF
    #!/bin/bash
    # auto-generated job file
    # generated from $PWD/$0
    # on ${DATE}
    #SBATCH --job-name=PAINTOR_${UIDN}_${ETH}_JOB
    #SBATCH --ntasks=1                          ##Number of PROCESSES
    #SBATCH --cpus-per-task=1                   ##Number of PROCESSES
    #SBATCH --mem-per-cpu=5000                  ##Memory specified for each core used (in MB) (no cores, use --mem=)
    #SBATCH -t 2-02:00:00                       ##Runtime in D-HH:MM:SS
    #SBATCH --share
    #SBATCH --partition=medium                  ## express(2h), short(12h), medium(2d2h), long(6d6h), interactive(2h)
    #
    #SBATCH --mail-user=${USER}@uab.edu
    #SBATCH --mail-type=ALL                     ## BEGIN, END, ERROR, ALL
    #
    #SBATCH --error=${LOG_DIR}/%j.%N.err.txt               ##File to which STDERR will be written
    #SBATCH --output=${LOG_DIR}/%j.%N.out.txt              ## File to which STDOUT will be written

Upon running, this section of the master script

LNGTH=`awk '{print NF}' ${TARG}/${UIDN}.annotations | tail -1`
        for NUM in \$( eval "echo {1..\$LNGTH}"); do
                ANNOT_COL=`head -1 ${TARG}/${UIDN}.annotations | awk -v NUM=\$NUM '{print \$NUM}'`
                if [ ! -f ${TEMP_DIR}/${UIDN}/Gname.Enrich.\$ANNOT_COL ]; then
                        $PAINTOR -input ${TARG}/input.files \
-Zhead ZSCORE.AFR,ZSCORE.EAS,ZSCORE.AFR \
-LDname AFR.LD,EAS.LD,EUR.LD \
-in ${TARG}/ \
-out $TEMP_DIR/${UIDN}/ \
-enumerate 2 \
-annotations \$ANNOT_COL \
-Gname Gname.Enrich.\$ANNOT_COL \
-Lname Lname.BF.\$ANNOT_COL
                fi
        done

prints the slave script as:

LNGTH=8134
    for NUM in $( eval "echo {1..$LNGTH}"); do
            ANNOT_COL=E001_15_coreMarks_mnemonics.bed.10_TssBiv.ES-I3_Cell_Line E001_15_coreMarks_mnemonics.bed.11_BivFlnk.ES-I3_Cell_Line E001_15_coreMarks_mnemonics.bed.12_EnhBiv.ES-I3_$
            if [ ! -f /data/scratch/vlaufer/PAINTOR3/temp/CLEAR_Okada_21_43755067_43955067/Gname.Enrich.$ANNOT_COL ]; then
                    /data/scratch/vlaufer/PAINTOR3/PAINTOR_V3.0/PAINTOR -input /data/scratch/vlaufer/PAINTOR3/Prepped_Input_Files/CLEAR_Okada_21_43755067_43955067/input.files -Zhead ZSCOR$
            fi
    done

Much of this appears to work, I believe the line that is failing is the line containing ANNOT_COL= . This line should only print the annotation corresponding to the column having value equal to $NUM (ie, if $NUM is 10, then the 10th column of the first line of the file ${TARG}/${UIDN}.annotations should be printed, not ALL columns. The part that is particularly confusing to me is that the LNGTH= line seems to work, and has very similar syntax.

I suspect the problem is with escaping characters, but I am not sure.

Answer 1

I reworked the section in question, it is now:

read -a ANNOT < ${TARG}/${UIDN}.annotations
for ANNOT_COL in "\${ANNOT[@]}"; do
        if [ ! -f ${TEMP_DIR}/${UIDN}/Gname.Enrich.\$ANNOT_COL ]; then
                echo "now generating marginal distribution for \$ANNOT_COL"
                $PAINTOR -input ${TARG}/input.files \\

The LNGTH and NUM variables have been removed, the script now iterates through the header of the ${TARG}/${UIDN}.annotations file directly.