how to efficiently cythonize the “vectorize” function (numpy library) - python

Question

as the title suggets, I'd like to efficiently cythonize the numpy.vectorize function, which, to the core, is simplyfying this piece below (the complete function is way too long to post but the majority of the time is spent here):

    def func(*vargs):
        for _n, _i in enumerate(inds):
            the_args[_i] = vargs[_n]
        kwargs.update(zip(names, vargs[len(inds):]))
        return self.pyfunc(*the_args, **kwargs)

I have read these guides ( http://cython.readthedocs.io/en/latest/src/tutorial/numpy.html and http://pandas.pydata.org/pandas-docs/stable/enhancingperf.html ) which are very useful but my knowledge of C is way too narrow to use them to a fraction of their potential.

how would you go about it ? [Python 3.5.1, Cython 0.25a, Numpy 1.10.4]

Answer 1

The function you show is just a bit of dancing to deal with kwargs . Note the comment at the head of that block in vectorize.__call__ . With simpler arguments it just sets func = self.pyfunc .

The actual work is done in the last line:

self._vectorize_call(func=func, args=vargs)

which does

outputs = ufunc(*inputs)
< return dtype conversion >

ufunc is, in most cases, frompyfunc(func, len(args), nout) .

So stripped of all this Python cover, it comes down to

np.frompyfunc(your_func, n, m)(args)

and frompyfunc is a compiled function. I suspect that function uses nditer (the c version) to broadcast the arguments, and feed the values as scalars to your_func . I discussed the use of nditer with cython in another recent SO.

In sum, as long as your_func is an impentrable (or general) python function, there's nothing cython can do to improve on this. The iteration is already being handled in compiled code.