Iterate over array of arrays with scipy.weave.inline

Question

I have a numpy.ndarray of dtype object containing exclusively other arrays of different length. I have C code, that does some computations with the nested arrays, but I'm not sure how to grab the inner array and it's size when iterating over it using the numpy C-API. So currently it looks something like this:

from scipy.weave import inline
import numpy as np

arrs = np.zeros(10, dtype=object)
for i in xrange(10):
    arrs[i] = np.arange(i*i)

for arr in arrs:
    inline(ccode, ['arr', 'other', 'args'])

I know, that it's not an optimal structure, but neither would be sparse matrices I guess. arrs is quite lengthy, about 100k, so including this python loop into C would be a great speedup, as it would eliminate the overhead of calling inline all the time. But how do I get arr in an iterated way from within C?

Answer 1

So I finally figured out how to do it. scipy.weave seems not to like object arrays, so first I converted it to a list. Then the list items can be grabbed using the python C-API. The object conversion is directly taken from some other precompiled inlined C-Code.

arrs = list()
for i in xrange(5):
    arrs.append(np.arange(i * i, dtype=int))
code = r"""
    long arrs_size = PyList_Size(arrs);
    for (long i=0; i<arrs_size; i++) {
        PyArrayObject* arr_array = convert_to_numpy(PyList_GetItem(arrs,i), "arr");
        conversion_numpy_check_type(arr_array,PyArray_LONG, "arr");
        npy_intp* Narr = arr_array->dimensions;
        npy_intp* Sarr = arr_array->strides;
        int Darr = arr_array->nd;
        long* arr = (long*) arr_array->data;
        long arr_size = 1;
        for (int n=0; n<Darr; n++) arr_size *= Narr[n];
        for (int j=0; j<arr_size; j++) printf("%ld ", arr[j]);
        printf("\n");
    }
"""
inline(code, ['arrs'])