Passing a structre from Python to C++ OR passing a numpy array instead of file address to Yolo

Question

I am trying pass in-memory images to Yolo rather than file addresses.

In the source code it reads the image using openCV. Since openCV return a mat format in c++ then it converts mat format to image format using mat_to_image(mat im) function. But opencv in python uses numpy.ndarray so I cant use mat_to_image() .

So I tried to put the numpy array in image format myself following the authors code here we have:

class IMAGE(ctypes.Structure):
_fields_ = [("w", ctypes.c_int),
            ("h", ctypes.c_int),
            ("c", ctypes.c_int),
            ("data", ctypes.POINTER(ctypes.c_float))]

Here is what I have:

import darknet as dn

im = cv2.imdecode(in-memory_bytelike_object, cv2.IMREAD_COLOR)
h,w,c = im.shape
my_image = IMAGE()
my_image.w = ctypes.c_int(w)
my_image.h = ctypes.c_int(h)
my_image.c = ctypes.c_int(c)
my_image.data = im.ctypes.data_as(ctypes.POINTER(ctypes.c_float))

print(dn.detect(net, meta, my_image))

I also changed detect function in this way:

def detect(net, meta, image, thresh=.5, hier_thresh=.5, nms=.45):
    # im = load_image(image, 0, 0)  //image_address->mat (using cv2)-> image (using mat_to_image) and return image 
    im = image
    ...

but when I run it I get this error:

ArgumentError: argument 2: <class 'TypeError'>: expected IMAGE instance instead of IMAGE

I guess the problem is that I am not passing the data structure in the right way but I'm not sure.

I read some other answers about passing a data structure between Python and C++ but I think in my case there should be a simpler solution.

Answer 1

Well I managed to do it in this way:

here is how you can convert a numpy array to image format:

import darknet as dn
def array_to_image(arr):
    arr = arr.transpose(2,0,1)
    c = arr.shape[0]
    h = arr.shape[1]
    w = arr.shape[2]
    arr = (arr/255.0).flatten()
    data = dn.c_array(dn.c_float, arr)
    im = dn.IMAGE(w,h,c,data)
    return im

Use the modified version of detect function as mentioned in the question.

so it goes like this:

im = cv2.imdecode(in-memory_bytelike_object, cv2.IMREAD_COLOR)
im = array_to_image(arr)
dn.rgbgr_image(im)
print(dn.detect(net, meta, im))

got the function from here