I'm writing a Python C extension by SWIG but getting frustrated when passing binary buffer to C api functions. Here's my example:
In utils.c
#include "utils.h"
void my_print_hexbytes(uint32_t *bytes, uint32_t bytes_len)
{
uint32_t i;
for(i = 0; i < bytes_len; i++)
printf("%02x", bytes[i]);
printf("\n");
}
In utils.h
#include "commons.h"
#ifndef XXXX_UTILS_H
#define XXXX_UTILS_H
void my_print_hexbytes(uint32_t *bytes, uint32_t bytes_len);
#endif /* XXXX_UTILS_H */
In commons.h
#ifndef XXXX_COMMONS_H
#define XXXX_COMMONS_H
....
....
#include <stdint.h>
....
....
#endif
In xxxx_for_py.i
%module xxxx_for_py
%{
#define SWIG_FILE_WITH_INIT
#include "commons.h"
#include "utils.h"
%}
%include "stdint.i"
void my_print_hexbytes(uint32_t *bytes, uint32_t bytes_len);
After compiling out _xxxx_for_py.so
, I gave a try testing it in IPython:
In [1]: import xxxx_for_py
In [2]: from ctypes import *
In [3]: a_t = c_uint * 2
In [4]: a = a_t(0x1, 0x2)
In [5]: xxxx_for_py.my_print_hexbytes(a, 2)
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-5-3c023fcf8b04> in <module>()
----> 1 xxxx_for_py.my_print_hexbytes(a, 2)
TypeError: in method 'my_print_hexbytes', argument 1 of type 'uint32_t *'
That time I had no idea how to handle this case. I tried changing the a
as bytearray
but it didn't work.
Not sure if anyone who could help me on this issue. Thanks!
Thanks for @Petesh comment. I've hit a try by casting a
to POINTER(c_uint)
but still not work :(
In [5]: xxxx_for_py.my_print_hexbytes(cast(a, POINTER(c_uint)), 2)
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-5-2abb9dae484d> in <module>()
----> 1 xxxx_for_py.my_print_hexbytes(cast(a, POINTER(c_uint)), 2)
TypeError: in method 'my_print_hexbytes', argument 1 of type 'uint32_t *'
Referenced by Unbounded Array section in SWIG documentation, I added carray.i
to xxxx_for_py.i
In xxxx_for_py.i
....
....stdint.i"
%include "carrays.i"
%array_class(uint32_t, uint32Array)
....
After re-compiling and loading .so
, it still got same error
In [1]: import xxxx_for_py
In [2]: a = xxxx_for_py.uint32Array(2)
In [3]: a[0] = 0x0
In [4]: a[1] = 0x1
In [5]: xxxx_for_py.my_print_hexbytes(a, 2)
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-5-3c023fcf8b04> in <module>()
----> 1 xxxx_for_py.my_print_hexbytes(a, 2)
TypeError: in method 'my_print_hexbytes', argument 1 of type 'uint32_t *'
After a week later at that time, user Nethanel post an answer . Though it still did not work, it could help me find something interesting.
After brief testing, I found solution by myself. See this article for details http://au9ustine.github.io/wiki/crypto/cuda/swig.html#typeerror-in-method-xxxx-argument-y-of-type-zzzz ( Update on 2017-01-13 : This page is broken, I'll move that page content here)
Keeping trying on testing Python codes on passing binary buffer to a trivial C module (ie library compiled by gcc without extra dependencies). And internals might look like:
integer list -> array/struct -> raw string -> C function parameter
From some descriptions from create_string_buffer . By create_string_buffer
, Python could provide such a feature dynamically allocating pieces of memory for current variables for use.
#include <stdio.h>
#include <stdlib.h>
#include <stdint.h>
size_t
print_int_buf(void *src, size_t len)
{
size_t i = 0;
uint32_t *int_ptr = (uint32_t *)src;
for (i = 0; i < len; i++) {
printf("%p: %x\n", int_ptr + i, int_ptr[i]);
}
return i;
}
If compiled as a shared library named my_ext_lib.so
, the code could be loaded in Python
import ctypes
from ctypes import *
import array
my_ext = ctypes.CDLL('./my_ext_lib.so')
buf = create_string_buffer(array.array('I', range(0x10)).tostring())
my_ext_lib.print_int_buf(buf, 0x10)
And it did work.
Though ctype and SWIG are quite different approaches for interoperating integration between Python and C, I use ctype code here to elaborate the basic mechanism and its working way of passing parameters to a compiled module (both for Python module and trivial C shared library module).
Since the significant difference was the function interface (or method signature) from my view, why not change it to void *
instead of uint32_t *
? (Though it's not the best way, it could be still an interim solution for containment)
I replaced uint32_t *
with void *
for the method my_print_hexbytes
, and appended dependency cdata.i
(I'm not sure if it works but for safety it has been added). So the changes were listed below and eventually it displayed expected result.
utils.c
#include "utils.h"
...
void
my_print_hexbytes(void *bytes, size_t bytes_len)
{
size_t i = 0;
uint32_t *int_buf_ptr = (uint32_t *)bytes;
for(i = 0; i < bytes_len; i++)
printf("%02x", int_buf_ptr[i]);
printf("\n");
}
header utils.h
#include "commons.h"
#ifndef XXXX_UTILS_H
#define XXXX_UTILS_H
...
void my_print_hexbytes(void *bytes, size_t bytes_len);
#endif /* XXXX_UTILS_H */
xxxx_for_py.i
%module xxxx_for_py
%{
#define SWIG_FILE_WITH_INIT
#include "commons.h"
#include "utils.h"
%}
%include "stdint.i"
%include "carrays.i"
%include "cdata.i"
%array_class(uint32_t, uint32Array)
...
void my_print_hexbytes(void *bytes, size_t bytes_len);
Invoking method in Python, it would be like:
In [1]: import xxxx_for_py
In [2]: a = xxxx_for_py.uint32Array(0x10)
In [3]: for i in range(0x10):
...: a[i] = i
...:
In [4]: xxxx_for_py.my_print_hexbytes(a, 0x10)
000102030405060708090a0b0c0d0e0f
Found you question, because of same problem.
Unfortunately, it seems advanced types are not supported by SWIG: in http://www.swig.org/Doc3.0/Library.html#Library_carrays , after description of array_class macro, it is written: "When using this macro, type is restricted to a simple type name like int or float."
I also tried unsuccessfully to use "unsigned int".
In the end my workaround was to use C++ vectors instead of uint32_t pointers, here is relevant part of swig file:
%include "stdint.i"
%include "std_vector.i"
namespace std {
%template(vectoruint32) vector<uint32_t>;
};
part of h file:
#include <vector>
static uint32_t foo(std::vector<uint32_t> v) {return v[1] - v[0];}
python:
import example
l = [1, 2, 3, 4]
example.foo(l)
Hope this helps
I had a very simple case but still got this error. The fix from this question worked: Swig python - c++ how to use type int8_t
I added %include "stdint.i"
directly after the module def in the .i file
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.