在 Google Colaboratory 上运行 faiss

Question

I am trying to run the simple examples of Faiss on Google Colab but keep get kernel crash and restart. The error in the log is : Intel MKL FATAL ERROR: Cannot load libmkl_avx2.so or libmkl_def.so. This happens when using both CPU or GPU versions. Here is the way I installed Faiss on Google collab

!wget  https://anaconda.org/pytorch/faiss-cpu/1.5.1/download/linux-64/faiss-cpu-1.5.1-py36h6bb024c_1.tar.bz2
!tar xvjf faiss-cpu-1.5.1-py36h6bb024c_1.tar.bz2
!cp -r lib/python3.6/site-packages/* /usr/local/lib/python3.6/dist-packages/
!pip install mkl

The code that I am trying to run is :

 import numpy as np
    d = 64                           # dimension
    nb = 100000                      # database size
    nq = 10000                       # nb of queries
    np.random.seed(1234)             # make reproducible
    xb = np.random.random((nb, d)).astype('float32')
    xb[:, 0] += np.arange(nb) / 1000.
    xq = np.random.random((nq, d)).astype('float32')
    xq[:, 0] += np.arange(nq) / 1000

import faiss                   # make faiss available
index = faiss.IndexFlatL2(d)   # build the index
print(index.is_trained)
index.add(xb)                  # add vectors to the index
print(index.ntotal)

k = 4                          # we want to see 4 nearest neighbors
D, I = index.search(xb[:5], k) # sanity check
print(I)
print(D)
D, I = index.search(xq, k)     # actual search
print(I[:5])                   # neighbors of the 5 first queries
print(I[-5:])  

The crash happened on this line.

D, I = index.search(xq, k)     # actual search

Any ideas?

Answer 1

Solution which worked for me:

!apt install libomp-dev
!python -m pip install --upgrade faiss faiss-gpu
import faiss

I took this code from here