tensorflow GPU on AWS elastic beanstalk - tf import error with calling “python” + “sudo” ( libcublas.so.9.0 error)

Question

I am getting the

"ImportError: libcublas.so.9.0: cannot open shared object file: No such file or directory"

error when I call my application with "sudo python3.6 ..." but not when I call just "python3.6 ..."

elastic beanstalk uses "sudo", so removing the command is not an option.

Only CUDA 9.0 is installed (which is the correct version for our environment), and /usr/local/cuda/lib64/libcublas.so.9.0 exists .

we created a separate ec2 instance with the same config and same AMI ID: CUDA9ClassicAmazonLinuxDLAMIwithMXNetTensorflowandCaffe2, and had no issues. It seems to be an issue present only with the elastic beanstalk.

$LD_LIBRARY_PATH = '/usr/local/cuda/lib64:/usr/local/lib:/usr/lib:/usr/local/cuda/extras/CUPTI/lib64:/usr/local/mpi/lib:/lib/:/home/ubuntu/src/caffe2/build:/home/ec2-user/src/caffe2/build:/usr/local/cuda/lib64:/usr/local/lib:/usr/lib:/usr/local/cuda/extras/CUPTI/lib64:/usr/local/mpi/lib:/usr/local/cuda/lib64:/usr/local/lib:/usr/lib:/usr/local/cuda/extras/CUPTI/lib64:/usr/local/mpi/lib:/lib/:/home/ubuntu/src/caffe2/build:/home/ec2-user/src/caffe2/build:/usr/local/cuda/lib64:/usr/local/lib:/usr/lib:/usr/local/cuda/extras/CUPTI/lib64:/usr/local/mpi/lib:'

$PATH = '/usr/local/cuda/bin:/usr/local/bin:/opt/aws/bin:/usr/local/mpi/bin:/home/ubuntu/src/caffe2/build:/home/ec2-user/src/caffe2/build:/usr/local/cuda/bin:/usr/local/bin:/opt/aws/bin:/usr/local/mpi/bin:/usr/local/cuda/bin:/usr/local/bin:/opt/aws/bin:/usr/local/mpi/bin:/home/ubuntu/src/caffe2/build:/home/ec2-user/src/caffe2/build:/usr/local/cuda/bin:/usr/local/bin:/opt/aws/bin:/usr/local/mpi/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/opt/aws/bin:/home/ec2-user/.local/bin:/home/ec2-user/bin'

any thoughts?

Answer 1

Add a file for setting the path to the new library(CUDA in your case) in

/etc/ld.so.conf.d/foo.conf

The contents of the file is the path to the CUDA library (/usr/local/cuda/lib64)

And call

 sudo ldconfig 

to make sure the library is included in the environment.