用boost.python交换numpy数组：pyublas或boost.numpy？

Question

I am interfacing a C++ data intense library with Python by py++ / boost.python . After profiling my program, I find 70% of the run time is spent on code like this:

ni = range(v2o.getHits())
tau = np.array([v2o.TofCorrectedTime[i] for i in ni])
q = [v2o.getCharge()[i] for i in ni]

v2o.TofCorrectedTime is typed __array_1_float_2368 from py++. v2o.getCharge() is typed _impl_details_range_iterator_ from py++, too. Size being about 2000, the convertion from these py++ array wrappers to numpy is slow:

In [42]: timeit np.array(v2o.TofCorrectedTime)
100 loops, best of 3: 2.52 ms per loop

In [43]: timeit np.array(v2o.getCharge())
100 loops, best of 3: 4.94 ms per loop

In [44]: timeit np.array([0]*2368)
1000 loops, best of 3: 310 µs per loop

In [45]: timeit np.array(np.zeros(2368))
100000 loops, best of 3: 4.41 µs per loop

I searched the web for a solution. The candidates are:

1. Cython with memoryview
2. pyublas
3. Boost.NumPy

Questions and Answers (updated):

• Is cython/memoryview easy to be integrated with boost.python and py++? I want to keep the rest of the library wrapper.

  No. (Jim's answer)

  cython c++ wrapper and boost.python have intrinsicly different infrastructures. It's hard for them to talk to each other. (Although in principle, we could teach py++ to output cython code. But that's another story.)

  Extending the present wrapper with Boost.NumPy is the most manageable way.

• Which one best suites my problem in terms of convertion overhead?

  (No definite answer yet.)

Thanks

Answer 1

(Disclaimer: I'm the primary author of Boost.NumPy.)

I'm afraid none of these options are particularly great. Here's how I think the pro/con analysis goes:

• Cython has a large number of users and developers, and hence you'll have a lot more support if you go with that option. It's not at all integrated with Boost.Python, however, and I think it'd be a tremendous amount of work to make Cython objects talk to Boost.Python, let alone Py++; you'd probably need to gain a pretty solid understanding of the low-level implementation details of both Cython and Boost.Python to get that going. You'd probably be better off scrapping your Py++/Boost.Python wrappers if you want to use Cython.

• Boost.NumPy has a much smaller community, and support resources are hence more limited, but it's much better suited to the code you already have. Py++ knows nothing about Boost.NumPy, so it won't automatically generate code that uses it (it might be that you could teach Py++ about Boost.NumPy; I'm not familiar enough with Py++ to know), but it's very straightforward to add custom Boost.Python code (and hence Boost.NumPy code) to a Py++ project.