问：python（在集群的bash脚本中使用cutadapt）：libpython3.7m.so.1.0：无法打开共享对象文件：无此类文件或目录

Question

I'm running a bash script on a cluster in which the program cutadapt (which utilizes python) is used.

Before I run my script, I loaded python version 3.7.0 and cutadapt

alias python=python3
module load python/3.7.0
pip3 install --user --upgrade cutadapt

and then send it off via qsub but the job keeps aborting early & I keep getting the following error:

/u/local/apps/python/3.7.0/bin/python3: 
error while loading shared libraries: libpython3.7m.so.1.0: 
cannot open shared object file: No such file or directory

It doesn't seem to be an issue with the $LD_LIBRARY_PATH since echo $LD_LIBRARY_PATH shows that the /u/local/apps/python/3.7.0/lib is preferentially listed.

Also, ldd /u/local/apps/python/3.7.0/bin/python3 shows that libpython3.7m.so.1.0 is found and points to:

libpython3.7m.so.1.0 => 
/u/local/apps/python/3.7.0/lib/libpython3.7m.so.1.0 (0x00007f66e01f6000) 

Additionally, calling cutadapt in the same manner as my script appears to be working properly:

~/.local/bin/cutadapt

This is cutadapt 2.3 with Python 3.7.0
Command line parameters:

Run "cutadapt --help" to see command-line options.
See https://cutadapt.readthedocs.io/ for full documentation.

My peer has run pretty much the exact same code and script and has had no issues with the script aborting/this error message.

Thank you in advance for any suggestions as to how I can troubleshoot this issue!

Edit (Thank you for your feedback!):

This is the script I submitted (which is nearly identical to my peer's aside from a different home directory on the cluster and different output directory):

#-- trim00s.sh --#
#!/bin/bash
#runs cutadapt on files 01-09

for i in {1..9}
do
fastq="Index0${i}.for.fq"
trimmedFastq="Index0${i}_trimmed.for.fq"
/u/home/e/evanderw/.local/bin/cutadapt -a GATCGGAAGAGCACACGTCTGAACTCCAGTCACNNNNNNATCTCGTATGCCGTCTTCTGCTTG -a "A{10}" -a "T{10}" -m 15 -q 30 -o ../../04_trimmed/lane_2/$trimmedFastq $fastq
done

The $LD_LIBRARY_PATH is the following after I first log onto the cluster (I'm guessing it's automatically set to these libraries as a default for this cluster?):

echo $LD_LIBRARY_PATH
/u/local/compilers/intel-cs/2013.0.028/mpi/intel64/lib:
/u/local/compilers/intel-cs/2013.0.028/itac/intel64/itac/slib_impi4:
/u/local/compilers/intel-cs/2013.0.028/composer_xe/compiler/lib/intel64:
/opt/intel/mic/coi/host-linux-release/lib:
/opt/intel/mic/myo/lib:
/u/local/compilers/intel-cs/2013.0.028/composer_xe/mpirt/bin/intel64:
/u/local/compilers/intel-cs/2013.0.028/composer_xe/ipp/lib/intel64:
/u/local/compilers/intel-cs/2013.0.028/composer_xe/mkl/lib/intel64:
/u/local/compilers/intel-cs/2013.0.028/composer_xe/tbb/lib/intel64

But then it changes after I have done alias python=python3 and module load python/3.7.0 to:

echo $LD_LIBRARY_PATH
/u/local/apps/python/3.7.0/lib:
/u/local/compilers/intel/17.0.1/compilers_and_libraries_2017.1.132/linux/mkl/lib/intel64:
/u/local/apps/xz/5.2.3/lib:
/u/local/apps/openssl/1.1.1/lib:
/u/local/apps/mpfr/3.1.5/gcc-4.9.3/lib:
/u/local/apps/hdf5/1.8.18/gcc-4.4.7-shared/lib:
/u/local/compilers/gcc/4.9.3/lib:
/u/local/compilers/gcc/4.9.3/lib64:
/u/local/compilers/intel-cs/2013.0.028/mpi/intel64/lib:
/u/local/compilers/intel-cs/2013.0.028/itac/intel64/itac/slib_impi4:
/u/local/compilers/intel-cs/2013.0.028/composer_xe/compiler/lib/intel64:
/opt/intel/mic/coi/host-linux-release/lib:
/opt/intel/mic/myo/lib:
/u/local/compilers/intel-cs/2013.0.028/composer_xe/mpirt/bin/intel64:
/u/local/compilers/intel-cs/2013.0.028/composer_xe/ipp/lib/intel64:
/u/local/compilers/intel-cs/2013.0.028/composer_xe/mkl/lib/intel64:
/u/local/compilers/intel-cs/2013.0.028/composer_xe/tbb/lib/intel64

Answer 1

I updated the script to ensure that the shebang was the first line & then printed off the $LD_LIBRARY_PATH as a text file:

#!/bin/bash

#-- trim00s.sh --#
#runs cutadapt on files 01-09

echo $LD_LIBRARY_PATH >foo.txt

for i in {1..9}
do
fastq="Index0${i}.for.fq"
trimmedFastq="Index0${i}_trimmed.for.fq"
/u/home/e/evanderw/.local/bin/cutadapt -a GATCGGAAGAGCACACGTCTGAACTCCAGTCACNNNNNNATCTCGTATGCCGTCTTCTGCTTG -a "A{10}" -a "T{10}" -m 15 -q 30 -o ../../04_trimmed/lane_2/$trimmedFastq $fastq
done

The error still persisted & the resulting foo.txt file was completely blank, meaning $LD_LIBRARY_PATH was reset to blank while the script was running via qsub .

So, I included a line in my script to now add the directory for the library my code was having problems finding to my $LD_LIBRARY_PATH & it worked!

#!/bin/bash

#-- trim00s.sh --#
#runs cutadapt on files 01-09

export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/u/local/apps/python/3.7.0/lib


for i in {1..9}
do
fastq="Index0${i}.for.fq"
trimmedFastq="Index0${i}_trimmed.for.fq"
/u/home/e/evanderw/.local/bin/cutadapt -a GATCGGAAGAGCACACGTCTGAACTCCAGTCACNNNNNNATCTCGTATGCCGTCTTCTGCTTG -a "A{10}" -a "T{10}" -m 15 -q 30 -o ../../04_trimmed/lane_2/$trimmedFastq $fastq
done