如何在python3中使用ctypes导入ostringstream？

Question

I am writing analyser for natural language and I have a wrapper of c++ code in python 3 created with swig . I'd like to use a function which is some kind of stream writer and it takes std::ostream & os as the parameter. So I guess it would work if I somehow import ostringstream (read as which lib.so I should use in my ctypes.CDLL ) in my python code then pass it to this function, lest call it create_stream_writer(stream) , and then use stream.str() to get string. Is it any way to do this using ctypes or any other library? I am using docker container running Ubuntu 18.04, python3.6

code should look like this I guess:

def analyse(text, config):
    reader = PlainTextReader.create_string_reader(text, config)
    stream = ctypes.ostringstream() # some magic hear
    writer = TokenWriter.create_stream_writer('plain', stream, reader.tagset())

    for sentence in sentences(reader):
        writer.write_sentence(sentence)
    return stream.str()

Answer 1

You can do this (and make it nice for Python developers too). This answer is essentially a Python 3 version of my older answer on wrapping iostreams .

To simplify things here I used boost's iostreams library. If you can't/don't use boost then you can write this all from standard C++ library components, it's just far more verbose.

I've also aimed higher than mapping io.StringIO to std::stringstream and instead gone for mapping any 'file like' Python object to any iostream . That is to say we use aim to use duck typing on the Python object to just call read() and write() sensibly as and when needed for our C++ stream objects.

%module test

%{
#include <boost/iostreams/stream.hpp>
#include <boost/iostreams/categories.hpp> 

// This is just a helper that we can use with boost iostreams to proxy everything to a Python object    
struct python_stream_device {
  typedef char char_type;
  typedef boost::iostreams::bidirectional_device_tag category;

  std::streamsize read(char* s, std::streamsize n) {
    PyObject *p = PyObject_CallMethod(o, "read", "l", static_cast<long int>(n));
    if (PyErr_Occurred()) {
      // TODO: throw a C++ exception to back out of wherever we are and then re-throw the Python one...
      assert(false);
    }
    assert(p);
    char *ptr = nullptr;
    Py_ssize_t len = 0;
    PyObject *str = PyUnicode_AsUTF8String(p);
    PyBytes_AsStringAndSize(str, &ptr, &len);
    if (PyErr_Occurred()) {
      assert(false); // Let's just pretend this is error handlng...
    }

    memcpy(s, ptr, len);
    Py_DECREF(str);
    Py_DECREF(p);
    return len;
  }

  std::streamsize write(const char* s, std::streamsize n) {
    PyObject *ret = PyObject_CallMethod(o, "write", "s#", s, static_cast<Py_ssize_t>(n));
    if (PyErr_Occurred()) {
      // See above
      assert(false);
    }
    std::streamsize r = PyLong_AsSsize_t(ret);
    Py_DECREF(ret);
    return r;
  }

  // Using this means we can rely on the default synthesised operator= + copy ctor etc. and saves us some code.
  swig::SwigPtr_PyObject o;

  python_stream_device(PyObject *o) : o(o) {}
};

typedef boost::iostreams::stream<python_stream_device> python_stream;

%}

// Here is the stuff that wraps it neatly
%typemap(in) std::iostream& (python_stream tmp) {
  // Writing the typemap this way lets us get RAII semantics despite the goto in the SWIG macros in the simplest way
  tmp.open(python_stream_device($input));  
  $1 = &tmp;
}

// We can just use the same typemaps for other cases too:
%apply std::iostream& { std::istream&, std::ostream& }; 


// Below is just for testing:    
%{
#include <iostream>
%}

%inline %{
  // This is the function you want to call
  void fun1(std::ostream& out) {
    assert(out.good());
    out << "Hello world, from C++";
    assert(out.good());
  }

  // This one is here for completeness because once you've got this far you may as well support this too.
  void fun2(std::istream& in) {
    std::string tmp;
    //in >> tmp;
    std::getline(in, tmp);
    assert(in.good());
    std::cout << "fun2 got: " << tmp << std::endl;
  }
%}

This is enough that you can then use some Python like this:

import io
import test

i=io.StringIO()
test.fun1(i)
print('After fun1: %s' % i.getvalue())

i=io.StringIO('hello world, from Python!\n')
test.fun2(i)

Answer 2

As Mark Tolonen pointed in comments it is impossible to do this with ctypes. So I just wrote c++ function that do everything I need, and then created a wrapper using SWIG. Because using typemap in SWIG to map StringIO(Python) to ostreingstream(C++) looks like black magic and I couldn't find the way to do this.