使用 swig，如何将二进制数据从 python 传递到 C/C++？

Question

If I have a C or C++ function like this in my my_module.h:

void my_function(const char* data, int len);

And my_module.c:

#include "my_module.h"

#include <unistd.h>

void my_function(const char* data, int len)
{
    // do fancy things with data
    write(1, data, len);
}

I will create a my_module.i for swig like this:

%inline %{
#include "my_module.h"
%}
%include "my_module.h"

I build like this:

swig -python -module my_module my_module.i

gcc -shared -fPIC my_module_wrap.c my_module.c -I/usr/include/python3.8 -lpython3.8 -o _my_module.so

Now from python I want to do this:

import my_module

in_file = open("my_binary_file", "rb")
bytes = in_file.read()
in_file.close()

my_module.my_function(bytes, len(bytes))

But I get:

TypeError: in method 'find_mistakes', argument 1 of type 'char const *'

I inspected type of the variable bytes

>>> type(bytes)
<class 'bytes'>

I don't know what that means. How do I just get the raw data to pass to C?

I don't want to convert the bytes to string, because it is not text. And when I tried to have convert it anyway the C side received text that encodes the binary instead of the raw binary. Something that looks like this:

b'\x\x00\n'

Edit:

Reading the manual I found this:

The char * datatype is handled as a NULL-terminated ASCII string. SWIG maps this into a 8-bit character string in the target scripting language. SWIG converts character strings in the target language to NULL terminated strings before passing them into C/C++. The default handling of these strings does not allow them to have embedded NULL bytes. Therefore, the char * datatype is not generally suitable for passing binary data. However, it is possible to change this behavior by defining a SWIG typemap. See the chapter on Typemaps for details about this.

So swig says char* is good for text, but not for binary data. What is the alternative to char* that works then?

It suggests changing swig's default behavior using typemaps. Is this the only way? If so, how is it done? And will it be a different hack for each language?

Answer 1

After many hours of frustration, I finally did it. Hopefully this answer saves other people some time and frustration.

Note that this answer works for python 3. I don't know how things would work with python 2.

From the docs :

In some cases, users may wish to instead handle all byte strings as bytes objects in Python 3. This can be accomplished by adding SWIG_PYTHON_STRICT_BYTE_CHAR to the generated code:

This means you just need this in your interface file:

%begin %{
#define SWIG_PYTHON_STRICT_BYTE_CHAR
%}

This will modify the behavior so that only Python 3 bytes objects will be accepted and converted to a C/C++ string, and any string returned from C/C++ will be converted to a bytes object in Python.

Bonus:

If you don't want to pass both the binary and the length from python you can do this in the interface file:

%include "typemaps.i"

// change "(const char* data, int len)" to match your functions declaration
%apply (char *STRING, size_t LENGTH) { (const char* data, int len) }

%include "my_module.h"

Now from python you can do this:

my_module.my_function(bytes) instead of my_module.my_function(bytes, len(bytes))

So my final interface file looks like this:

%module my_module

%begin %{
#define SWIG_PYTHON_STRICT_BYTE_CHAR
%}

%inline %{
#include "my_module.h"
%}

%include "typemaps.i"

%apply (char *STRING, size_t LENGTH) { (const char* data, int len) }

%include "my_module.h"