如何将 NumPy 数组与 ctypes 一起使用？

Question

I am still writing on a python interface for my c code with ctypes. Today I substituted my file reading function with a python version, which was programmed by somebody else using NumPy. The 'old' c version was called with a byref(p_data) while p_data=PFloat() (see below). The main function takes the p_data .

Old file reading:

p_data=POINTER(c_float)
foo.read(filename,byref(p_data))
result=foo.pymain(p_data)

The python file reading function, on the other hand, returns a NumPy array. My question now is:

How do I convert a NumPy array to POINTER(c_float) ?

I googled but only found the other way around: C arrays through ctypes accessed as NumPy arrays and things I didn't understand: C-Types Foreign Function Interface (numpy.ctypeslib)

[update] corrected a mistake in the example code

Answer 1

Your code looks like it has some confusion in it -- ctypes.POINTER() creates a new ctypes pointer class , not a ctypes instance. Anyway, the easiest way to pass a NumPy array to ctypes code is to use the numpy.ndarray 's ctypes attribute's data_as method. Just make sure the underlying data is the right type first. For example:

import ctypes
import numpy
c_float_p = ctypes.POINTER(ctypes.c_float)
data = numpy.array([[0.1, 0.1], [0.2, 0.2], [0.3, 0.3]])
data = data.astype(numpy.float32)
data_p = data.ctypes.data_as(c_float_p)

Answer 2

Using np.ndarrays as ctypes arguments

The preferable approach is using ndpointer , as mentioned in the numpy-docs .

This approach is more flexible than using, for example, POINTER(c_double), since several restrictions can be specified, which are verified upon calling the ctypes function. These include data type, number of dimensions, shape and flags. If a given array does not satisfy the specified restrictions, a TypeError is raised.

Minimal, Reproducible Example

Calling memcpy from python. Eventually the filename of the standard C-library libc.so.6 needs to be adjusted.

import ctypes
import numpy as np

n_bytes_f64 = 8
nrows = 2
ncols = 5

clib = ctypes.cdll.LoadLibrary("libc.so.6")

clib.memcpy.argtypes = [
    np.ctypeslib.ndpointer(dtype=np.float64, ndim=2, flags='C_CONTIGUOUS'),
    np.ctypeslib.ndpointer(dtype=np.float64, ndim=1, flags='C_CONTIGUOUS'),
    ctypes.c_size_t]
clib.memcpy.restype = ctypes.c_void_p

arr_from = np.arange(nrows * ncols).astype(np.float64)
arr_to = np.empty(shape=(nrows, ncols), dtype=np.float64)

print('arr_from:', arr_from)
print('arr_to:', arr_to)

print('\ncalling clib.memcpy ...\n')
clib.memcpy(arr_to, arr_from, nrows * ncols * n_bytes_f64)

print('arr_from:', arr_from)
print('arr_to:', arr_to)

Output

arr_from: [0. 1. 2. 3. 4. 5. 6. 7. 8. 9.]
arr_to: [[0.0e+000 4.9e-324 9.9e-324 1.5e-323 2.0e-323]
 [2.5e-323 3.0e-323 3.5e-323 4.0e-323 4.4e-323]]

calling clib.memcpy ...

arr_from: [0. 1. 2. 3. 4. 5. 6. 7. 8. 9.]
arr_to: [[0. 1. 2. 3. 4.]
 [5. 6. 7. 8. 9.]]

If you modify the ndim=1/2 arguments of ndpointer to be inconsistent with the dimensions of arr_from/arr_to , the code fails with an ArgumentError .

As the title of this question is quite general, ...

Constructing a np.ndarray from a ctypes.c_void_p result

Minimal, Reproducible Example

In the following example, some memory is allocated by malloc and filled with 0s by memset . Then a numpy array is constructed, to access this memory. Of course the occur some ownership issues, as python will not free memory, which was allocated in c. To avoid memory leaks , one has to free the allocated memory again by ctypes. The copy method can be used for the np.ndarray to acquire ownership .

import ctypes
import numpy as np

n_bytes_int = 4
size = 7

clib = ctypes.cdll.LoadLibrary("libc.so.6")

clib.malloc.argtypes = [ctypes.c_size_t]
clib.malloc.restype = ctypes.c_void_p

clib.memset.argtypes = [
    ctypes.c_void_p,
    ctypes.c_int,
    ctypes.c_size_t]
clib.memset.restype = np.ctypeslib.ndpointer(
    dtype=np.int32, ndim=1, flags='C_CONTIGUOUS')

clib.free.argtypes = [ctypes.c_void_p]
clib.free.restype = ctypes.c_void_p


pntr = clib.malloc(size * n_bytes_int)
ndpntr = clib.memset(pntr, 0, size * n_bytes_int)
print(type(ndpntr))
ctypes_pntr = ctypes.cast(ndpntr, ctypes.POINTER(ctypes.c_int))
print(type(ctypes_pntr))
print()
arr_noowner = np.ctypeslib.as_array(ctypes_pntr, shape=(size,))
arr_owner = np.ctypeslib.as_array(ctypes_pntr, shape=(size,)).copy()
# arr_owner = arr_noowner.copy()


print('arr_noowner (at {:}): {:}'.format(arr_noowner.ctypes.data, arr_noowner))
print('arr_owner (at {:}): {:}'.format(arr_owner.ctypes.data, arr_owner))

print('\nfree allocated memory again ...\n')
_ = clib.free(pntr)

print('arr_noowner (at {:}): {:}'.format(arr_noowner.ctypes.data, arr_noowner))
print('arr_owner (at {:}): {:}'.format(arr_owner.ctypes.data, arr_owner))

print('\njust for fun: free some python-memory ...\n')
_ = clib.free(arr_owner.ctypes.data_as(ctypes.c_void_p))

print('arr_noowner (at {:}): {:}'.format(arr_noowner.ctypes.data, arr_noowner))
print('arr_owner (at {:}): {:}'.format(arr_owner.ctypes.data, arr_owner))

Output

<class 'numpy.ctypeslib.ndpointer_<i4_1d_C_CONTIGUOUS'>
<class '__main__.LP_c_int'>

arr_noowner (at 104719884831376): [0 0 0 0 0 0 0]
arr_owner (at 104719884827744): [0 0 0 0 0 0 0]

free allocated memory again ...

arr_noowner (at 104719884831376): [ -7687536     24381 -28516336     24381         0         0         0]
arr_owner (at 104719884827744): [0 0 0 0 0 0 0]

just for fun: free some python-memory ...

arr_noowner (at 104719884831376): [ -7687536     24381 -28516336     24381         0         0         0]
arr_owner (at 104719884827744): [ -7779696     24381 -28516336     24381         0         0         0]