Cython 用零初始化矩阵

Question

Description

I simply want to create a matrix of rows x cols that is filled with 0 s. Always working with numpy I thought using np.zeros as describedin the docs is the easiest:

DTYPE = np.int
ctypedef np.int_t DTYPE_t

def f1():
    cdef:
        int dim = 40000
        int i, j
        np.ndarray[DTYPE_t, ndim=2] mat = np.zeros([40000, 40000], dtype=DTYPE)
        
    for i in range(dim):
        for j in range(dim):
            mat[i, j] = 1

Then I compared this using the arrays in c:

def f2():
    cdef:
        int dim = 40000
        int[40000][40000] mat
        int i, j
    
    for i in range(dim):
        for j in range(dim):
            mat[i][j] = 1

The numpy version took 3 secs on my pc whereas the c version only took 2.4e-5 secs . However when I return the array from f2() I noticed it is not zero filled (of course here it can't be, i==j however when not filling it it won't return a 0 array either). How can this be done in cython. I know in regular C it would be like: int arr[n][m] = {}; .

Question

How can the c array be filled with 0s ? (I would go for numpy instead if there is something obvious wrong in my code)

Answer 1

You do not want to be writing code like this:

1. int[40000][40000] mat generates a 6 gigabyte array on the stack (assuming 4 byte ints). Typically maximum stack sizes are of the order of a few Mb. I have no idea how this isn't crashing your PC.

2. However when I return the array from f2() [...]

   The array you have allocated is completely local to the function. From a C point of view you cannot return it since it ceases to exist after the function has finished. I think Cython may convert it to a (nested) Python list for you. This requires a slow copy element-by-element and is not what you want.

For what you're doing here you're much better just using Numpy.

---

Cython doesn't support a good equivalent of the C arr = {} so if you do want initialize sensible, small C arrays you need to use of one:

1. loops,
2. memset (which you can cimport from libc.string ),
3. Create a typed memoryview of it and do memview[:,:] = 0

---

The numpy version took 3 secs on my pc whereas the c version only took2.4e-5 secs.

This kind of difference usually suggests that the C compiler has optimized some code out (by detecting that the result is unused). It is unlikely to be a genuine speed-up.