Could not load dynamic library 'libcudart.so.11.0' ? / failed call to cuInit: UNKNOWN ERROR (303)?

Question

I am a beginner TensorFlow user and am running into the following issue when attempting to load an already saved model for segmentation on a test images.

i installed all the libraries on a virtual environment that i created.

The same code runs on google colab and now i am trying to run it on my machine.

My Environment

Ubunutu 16 / tensorflow 2.5.0

My code

When running the code :

import os
from glob import glob
from tqdm import tqdm
import cv2
import tensorflow as tf
import nibabel as nib 
import numpy as np
import matplotlib.pyplot as plt

test_images = sorted(glob("/mnt/DATA2To/projet/all/Souris/SOD/data/20210726_C321/EXVIVO/IRM/test-20210726_C321/*"))   # images 160x120x1


i = 0 # iterator initialized to zero
model = tf.keras.models.load_model("/mnt/DATA2To/projet/all/Souris/SOD/segmentation/segmentation-moelle.h5", compile= False )
`
for path in tqdm(test_images, total=len(test_images)):                                                          
      x = nib.load(path)  # load the images (160x1x120)
      new_header = header=x.header.copy() # copy the header to a variable for writing the results at the end 
      x = nib.load(path).get_data() # get the data from the image loaded
      original_image = x
      original_image_bis = original_image.transpose((0,2,1))      
      h, w, _ = original_image_bis.shape 
      original_image_bis = cv2.resize(original_image_bis, (w, h)) 
      x = x.transpose((0,2,1)) # permute the image axes to (160x120x1)
      x = cv2.resize(x, (128, 128)) # resize the image to have a shape of (128x128) 
      x = (x - x.min()) / (x.max() - x.min()) # do the min-max normalisation
      x.shape= x.shape + (1,) # add the third axes (128x128x1)
      x = x.astype(np.float32) 
      
      x1 = np.expand_dims(x, axis=0) 
      pred_mask = model.predict(x1)[0] 
      
      #pred_mask = (np.where(pred_mask > np.mean(pred_mask), 1,0))  
      pred_mask = pred_mask.astype(np.float32) 
      pred_mask1 = cv2.resize(pred_mask, (w, h)) 
      pred_mask1 = (np.where(pred_mask1 > 0.92, 1,0))  
      pred_mask1.shape= pred_mask1.shape + (1,) # add the third axes (160x120x1)
      pred_mask1 = pred_mask1.transpose((0,2,1)) #permute the image axes to (160x1x120)
      Sform= new_header.get_base_affine() 
      pred_mask2 = nib.Nifti1Image(pred_mask1,None, header= new_header) 
      fname= "/mnt/DATA2To/projet/all/Souris/SOD/data/20210726_C321/EXVIVO/IRM/results-moelle/image%04d.nii" %i  
      nib.save(pred_mask2, fname)  
      i+=1

My Error

I am greeted with this error :

(venv) etudiant@PTT:~$ python3 '/home/etudiant/Documents/code/Segmentation.py'
2021-07-28 09:58:12.539200: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /mnt/software//mrtrix/lib::/opt/minc/lib:/opt/minc/lib/InsightToolkit
2021-07-28 09:58:12.539221: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
2021-07-28 09:58:13.429146: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcuda.so.1'; dlerror: libcuda.so.1: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /mnt/software//mrtrix/lib::/opt/minc/lib:/opt/minc/lib/InsightToolkit
2021-07-28 09:58:13.429164: W tensorflow/stream_executor/cuda/cuda_driver.cc:326] failed call to cuInit: UNKNOWN ERROR (303)
2021-07-28 09:58:13.429179: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:156] kernel driver does not appear to be running on this host (PTT): /proc/driver/nvidia/version does not exist
2021-07-28 09:58:13.429322: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
0it [00:00, ?it/s]

Can anyone would tell me what is wrong and how to fix that!?

Answer 1

W stands for "Warning" and I stands for "Information".

There are no problems with your code, TF just tells you it did not find the libraries required for GPU computation; this does not mean that TensorFlow does not run successfully on CPU.

What you can do instead to avoid receiving such messages in the future is to suppress the warnings.

Solution 1:

import os
os.environ['TF_CPP_MIN_LOG_LEVEL'] = '2'
import tensorflow as tf

Level 2 means to ignore warning and information, and print only the error.

Solution 2:

  import tensorflow as tf
  tf.logging.set_verbosity(tf.logging.ERROR) 

Solution 3:

  import logging
  tf.get_logger().setLevel(logging.ERROR)