Using Unicode in Boost Python with a C++ function that takes std::wstring

Question

I am using the Boost Python library to wrap a C++ class I have so that I can call its methods from Python. My C++ class Clazz has public methods:

void doSomething(std::string& s) { ... }
void doSomethingWide(std::wstring& ws) { ... }

I've created a BOOST_PYTHON_MODULE which points to these two methods. The first one which uses std::string I'm able to call fine. However, when I try to call the second one with a Python Unicode string:

x = u'hello'
Clazz.doSomethingWide(x)

I get the error:

ArgumentError: Python argument types in Clazz.doSomethingWide(Clazz, unicode) did not match C++ signature: doSomething(Clazz, std::wstring)

I had hoped that unicode would automatically interface with std::wstring as the regular Python string type did with std::string . However, this appears to not be the case.

In another thread, someone suggested first doing the conversion:

x = str(x.encode('utf-8'))

However, I am dealing with very large strings and this destroys the performance of my code since it is O(n) in the number of characters of x .

I do have the ability to modify the C++ library that I'm trying to interface with. Is there a way to pass in Python unicode types into my C++ library in a way that I can use them? I've searched the Internet far and wide and found some references to converters and other things, but implementing them did not fix the above error message (very possibly I'm not using them correctly).

Answer 1

In short, type conversions generally result in rvalue objects, and thus the parameters must either accept by value or by const reference. Hence, change:

void doSomethingWide(std::wstring&);

to either of the following:

void doSomethingWide(std::wstring);
void doSomethingWide(const std::wstring&);

---

Boost.Python added std::wstring conversions on 11-SEP-2003. As a general rule, when type conversions occur in Boost.Python, the resulting object is treated as an rvalue. This behavior is indirectly noted in the boost::python::extract observer specification:

Converts the stored pointer to result_type , which is either T or T const& .

If lvalue conversions were supported, it could introduce awkward semantics for some types. For example, immutable Python strings could be modified by C++ functions.

---

Here is a complete minimal example:

#include <iostream>
#include <string>
#include <boost/python.hpp>

class spam
{
public:
  void doSomething(const std::string& str)
  {
    std::cout << "spam::doSomething(): " << str << std::endl;
  }

  void doSomethingWide(const std::wstring& str)
  {
    std::wcout << "spam::doSomethingWide(): " << str << std::endl;
  }
};

BOOST_PYTHON_MODULE(example)
{
  namespace python = boost::python;
  python::class_<spam>("Spam")
    .def("doSomething", &spam::doSomething)
    .def("doSomethingWide", &spam::doSomethingWide)
    ;
}

Interactive usage:

>>> import example
>>> spam = example.Spam()
>>> spam.doSomething("test")
spam::doSomething(): test
>>> spam.doSomethingWide(u"test")
spam::doSomethingWide(): test