在没有 GIL 的情况下在 Cython 中并行化

Question

I'm trying to compute some columns of a numpy array, operating on python objects ( numpy array) in a for loop using a cdef function.

I would like to do it in parallel. But not sure how to.

Here is a toy example, one def function calls a cdef function in a for loop using prange , which is not allowed because np.ndarray is a python object. In my real problem, one matrix and one vector are the arguments of the cdef function, and some numpy matrix operations are performed, like np.linalg.pinv() (which I guess is actually the bottleneck).

%%cython
import numpy as np
cimport numpy as np
from cython.parallel import prange
from c_functions import estimate_coef_linear_regression

DTYPE = np.float
ctypedef np.float_t DTYPE_t

def transpose_example(np.ndarray[DTYPE_t, ndim=2] data):
    """
    Transposes a matrix. It does each row independently and parallel
    """

    cdef Py_ssize_t n = data.shape[0]
    cdef Py_ssize_t t = data.shape[1]

    cdef np.ndarray[DTYPE_t, ndim = 2] results = np.zeros((t, n))

    cdef Py_ssize_t i

    for i in prange(n, nogil=True):
        results[i, :] = transpose_vector(data[:, i])

    return results

cdef transpose_vector(np.ndarray[DTYPE_t, ndim=1] vector):
    """
    transposes a np vector
    """
    return vector.transpose()

a = np.random.rand(100, 20)
transpose_example(a)

outputs

Converting to Python object not allowed without gil

What would be the best way to do this in parallel?

Answer 1

You can pass typed memoryview slices ( cdef transpose_vector(DTYPE_t[:] vector) ) around without the GIL - it's one of the key advantages of the newer typed memoryview syntax over np.ndarray .

However,

• You can't call Numpy member functions (like transpose) on memoryviews, unless you cast back to a Numpy array ( np.asarray(vector) ). This requires the GIL.
• Calling any kind of Python function (eg transpose ) is going to require the GIL. This can be done inside a with gil: block, but when that block is almost your entire loop that becomes pretty pointless.
• You don't specify a return type for transpose_vector , and so it'll default to object , which requires the GIL. You could specify a Cython return type, but I suspect even returning a memoryview slice may require some reference counting somewhere.
• Be careful not to have multiple threads overwriting the same data in your passed memoryview slice.

In summary: memoryview slices, but bear in mind you're quite limited in what you can do without the GIL. Your current example just isn't parallelizable (but this may be mostly because it's a toy example).

Answer 2

Q : "What would be the best way to do this in parallel?"
+
" I have used intentionally np.transpose() to show that I have to use a python object. "

Let me start with freely paraphrasing the fabulous Henry FORD's maxim: the least defective part of an automobile is the very one, which is not there at all - it can never get damaged.

Those, who know how the numpy 's internal array-object representation works are sure,
it takes _almost zero time
and
it requires _almost zero memory-I/Os _{( at least for the last decade or so it did )}

---

WHY ?

Numpy is smart .
It does not move any data for this at all. It just adapts the indexing at a cost of about 15 [us]

---

ANSWER :

The Best Ever way to do the np.transpose() does not need any parallelisation at all.

Any attempt to do so will result in a way poorer performance, due to artificially enforced many useless memory-I/Os that the native np.transpose() never does - it just swaps indexing-scheme, without moving the heaps of cell-data, all rest in their original places (incl. keeping any and all cache-coherences valid - so any next access is going to take place 0.5 ~ 5 [ns] from cache, not having to pay again immense amounts of many times 150-350 [ns] for moving any cell-data from/to physical RAM-locations and devastating the cache-lines)

---

THE PROOF :

______1x THE PROBLEM SIZE 100 x 20 x 8 [B] ( 16 kB RAM).transpose() ~ 17 [us]

>>> r =   100; c =   20; a = np.arange(r*c).reshape( (r,c) );a.itemsize*a.size/1E6
0.016
>>> aClk.start(); _ = a.transpose(); aClk.stop()  ####  16   [kB] RAM-footprint
17                                                ####  17   [us] !!! ZERO-COPY

---

____100x THE PROBLEM SIZE 1000 x 200 x 8 [B] (1.6 MB RAM).transpose() ~ 17 [us]

>>> r =  1000; c =  200; a = np.arange(r*c).reshape( (r,c) );a.itemsize*a.size/1E6
1.6
>>> aClk.start(); _ = a.transpose(); aClk.stop()  ####   1.6 [MB] RAM-footprint
17                                                ####  17   [us] !!! ZERO-COPY

---

__10000x THE PROBLEM SIZE 10000 x 2000 x 8 [B] (160 MB RAM).transpose() ~ 16 [us]

>>> r = 10000; c = 2000; a = np.arange(r*c).reshape( (r,c) );a.itemsize*a.size/1E6
160.0
>>> aClk.start(); _ = a.transpose(); aClk.stop()  #### 160.0 [MB] RAM-footprint
16                                                ####  16   [us] !!! ZERO-COPY

Moving or ALAP-allocating & copying that many RAM-stored megabytes of cell-data, as does any prange -d or whatever else code would do, will take ages, not the smart 16 [ns] as the smart numpy-design does.

---

>>> a.flags
  C_CONTIGUOUS    : True <-------------------------ORIGINAL [indirect] RAM-indexing
  F_CONTIGUOUS    : False
  OWNDATA         : False
  WRITEABLE       : True
  ALIGNED         : True
  WRITEBACKIFCOPY : False
  UPDATEIFCOPY    : False
>>>
>>> _.flags
  C_CONTIGUOUS    : False
  F_CONTIGUOUS    : True <------------------------TRANSPOSE'd
  OWNDATA         : False
  WRITEABLE       : True
  ALIGNED         : True
  WRITEBACKIFCOPY : False
  UPDATEIFCOPY    : False

---

QED

---

EPILOGUE :

Now comes the point behind the notice clear and sound. Hopefully that.