pybind11: Can't get std::list to work in Python

Question

In my C++ project I have a method which returns std::list<ModelComponent*>* , where ModelComponent is a custom class I've defined.

My wrapper looks like this:

py::class_<ComponentManager>(m, "ComponentManager")
    .def(py::init<Model*>())
    .def("getAllComponents", &ComponentManager::getAllComponents,
        py::return_value_policy::reference);

When I try to use this method in Python I get the following error:

TypeError: Unable to convert function return value to a Python type! The signature was
        (self: libgenesys.ComponentManager) -> std::__cxx11::list<ModelComponent*, std::allocator<ModelComponent*> >

Did you forget to `#include <pybind11/stl.h>`? Or <pybind11/complex.h>,
<pybind11/functional.h>, <pybind11/chrono.h>, etc. Some automatic
conversions are optional and require extra headers to be included
when compiling your pybind11 module.

If I do #include <pybind11/stl.h> in this file though, all sorts of errors are thrown during compilation, starting with:

/usr/include/pybind11/cast.h:1408:73: error: no matching function for call to ‘get<0>(std::pair<ModelComponent*, unsigned int>*&)’
In file included from /usr/include/c++/9.2.0/bits/unique_ptr.h:36,
                 from /usr/include/c++/9.2.0/memory:80,
                 from /usr/include/c++/9.2.0/thread:39,
                 from main.cpp:15:

This error seems to be thrown because of another class (which also has a wrapper) that defines typedef std::pair<ModelComponent*, unsigned int> Connection; (even though I ignore the methods returning this particular type).

I can't seem to find a way to get the method returning a std::list to work, the documentation says including pybind11/stl.h should do the trick, but for me it's only throwing more errors.

Edit: The code on which std::pair<ModelComponent*, unsigned int> is used is posted here: https://pastebin.com/AX2XBYEd

Answer 1

The problem is your binding of std::list<Connection*>* getList() . SWIG doesn't know what to do with Connection* , only with Connection . You'll need to write a binding for Connection* .

Answer 2

In your pastebin code, you have:

typedef std::pair<ModelComponent*, unsigned int> Connection;

replace that with:

template<typename T1, typename T2>
struct MyPair {
    T1 first;
    T2 second;
};

typedef MyPair<ModelComponent*, unsigned int> Connection;

while leaving the #include "pybind11/stl.h" in place should do the trick.

At issue is that none of the pre-provided specializations that were written for conversions match. You can of course also provide your own specialization.

It's one of the reasons why in cppyy I prefer to leave objects as-is, and simply provide a pythonistic interface, and if you truly want a python object (a list as it may be here), it's trivially converted. Example below (I added inline definitions for class ConnectionManager to keep it simple; you hadn't any on pastebin), to compare:

>>> import ConnectionManager
>>> cm = ConnectionManager.ConnectionManager()
>>> cm.getList()
[]
>>>

versus:

>>> import cppyy
>>> cppyy.include("ConnectionManager.h")
>>> cm = cppyy.gbl.ConnectionManager()
>>> cm.getList()
<cppyy.gbl.std.list<pair<ModelComponent*,unsigned int>*> object at 0x7faa345759c0>
>>> list(_)
[]
>>> 

And because of the pythonizations, it's a drop-in replacement in for-loops etc. as well (it implements the __iter__ protocol, for example). Either which way, this approach certainly saves you from having to wade through those long C++ template error messages. :)