NumPy + BLAS + LAPACK on GPU (AMD and Nvidia)

Question

We have a Python code which involves expensive linear algebra computations. The data is stored in NumPy arrays. The code uses numpy.dot, and a few BLAS and LAPACK functions which are currently accessed through scipy.linalg.blas and scipy.linalg.lapack. The current code is written for CPU. We want to convert the code so that some of the NumPy, BLAS, and LAPACK operations are performed on a GPU.

I am trying to determine the best way to do this is. As far as I can tell, Numba does not support BLAS and LAPACK functions on the GPU. It appears that PyCUDA may the best route, but I am having trouble determining whether PyCUDA allows one to use both BLAS and LAPACK functions.

EDIT: We need the code to be portable to different GPU architectures, including AMD and Nvidia. While PyCUDA appears to offer the desired functionality, CUDA (and hence, PyCUDA) cannot run on AMD GPUs.

Answer 1

Have you tried looking into scikit-cuda? https://scikit-cuda.readthedocs.io/en/latest/#

It seems to use pyCUDA to access CUDA-toolkit libraries (CUBLAS, CUSOLVER, CUFFT, CULA) as well as provide their own implementation of some LAPACK routines based on CUBLAS.

I have used it for CUBLAS routines and it was a pleasant experience, I hope it would be the same for LAPACK

Answer 2

Another option is ArrayFire. While this package does not contain a complete BLAS and LAPACK implementation, it does offer much of the same functionality. It is compatible with OpenCL and CUDA, and hence, is compatible with AMD and Nvidia architectures. It has wrappers for Python, making it easy to use.