Converting C array to Python bytes

Question

I'm trying to use Cython to run a C++ library in python3 environment. When I try to return the int array back to python like this:

def readBytes(self, length):
    cdef int *buffer = []
    self.stream.read(buffer, length)
    return buffer 

I get the error

    return buffer
                ^
Cannot convert 'int *' to Python object

PS I don't get errors if I use

 cdef char *buffer = ''

Answer 1

it looks like stream.read() allocates the memory pointed by buffer . If this is the case, you cannot return to python space data allocated in C++ space. You should:

1) create a python object, or a numpy array if you prefer, in python/cython code

2) copy the data from the allocated memory pointed by *buffer to your new shiny python object allocated in python space. You can then return this object.

This is necessary because python cannot deal with the memory allocated in C space in any way, and the memory allocated by your C code is leaking, ie it will not be deallocated.

Now you also asked why you do not get the error with cdef char *buffer = '' . In this latter case, cython recognizes that buffer points to a string, and automatically generates a new python object with the content pointed by buffer . Example follows for ipython :

%%cython
def ReturnThisString():
    cdef char *buffer = 'foobar'
    return buffer

print ReturnThisString() #this outputs 'foobar'

Notice that buffer is initialized by your C compiler on the stack, and there's no guarantee that when you use this function from python the string will still be there at that memory position. However, when cython runs the return statement it automatically initializes a python string from your char * pointer. (In python 3 I guess it is converted to bytes as @Veedrac says, but this is a minor point). In this second case, the creation of the python object and the copy operation is hidden and taken care by cython , but it is still there.

Answer 2

char can be automatically coerced to bytes because Cython thinks that they're approximately the same and can do it fast. Note that char * pointers will be null terminated by default.

This is not implemented for int * . You will typically want to coerce to a Numpy object (this is actually wrapping the array). If you want something faster, think about cpython.array .