C++ & Python: Pass and return a 2D double pointer array from python to c++

Question

I want to pass a 2D array from Python to a C++ function and then return an array of the same type, same dimensions, to Python. I am aware this question has already been asked several times, but I haven't been able to find a relevant answer to my question. For my problem, I must use a double pointer array and have the function returning a double pointer array (not void as many examples show).

My C++ function is:

#include <stdio.h>      
#include <stdlib.h> 

extern "C" double** dot(double **a, int m, int n){

    double **arr = (double **)malloc(m * sizeof(double *)); 
    for (int i=0; i<m; i++) 
         arr[i] = (double*)malloc(n * sizeof(double));

    for (int i=0; i < m; i++){
        for (int j=0; j < n; j++){
            arr[i][j] = a[i][j];
            }
    }
    return arr;
}  

For the moment, I have used Ctypes . I know I could use the Swig interface but I would prefer avoiding it given that I don't know it very well. However, I am still open to any suggestion. My problem if I had to use Swig is that, if I'm not mistaking, I would have to use a Typemap in order to decompose the pointer structure, and it's a part I don't understand very well.

What I have tried for the moment in Python is:

import ctypes as c
import numpy as np

ty_ = np.ctypeslib._ctype_ndarray(c.POINTER(c.POINTER(c.c_double)), (3,3))
x = np.arange(9.).reshape(3,3)

_dll = ctypes.CDLL('./double_2D.so')

_foobar = _dll.dot
_foobar.argtype = type(y)
_foobar.restype = type(y)

d = _foobar(y, 3, 3) #I would like d to be a nice matrix like x 

I have also tried

c.cast(_foobar(y,3,3), c.POINTER(c.POINTER(c.c_double)))

But none of the examples above work. So therefore, any suggestion for defining the argtype or restype , or a snippet for Typemap in Swig would be of great help.

Answer 1

Listing [Python 3.Docs]: ctypes - A foreign function library for Python .

Couple of thoughts:

• " double pointer array " is misleading:
  • There is no array
  • " double pointer " might mean either pointer to double or pointer to pointer to something (including double )
• The solution with double pointers (to double :) ) seems a bit complex, (as also specified in the comments). I tend to think it's an XY Problem . Normally, one should only deal with simple pointers, especially if their knowledge in this area isn't very strong (and this seems to apply here, as I noticed from the other questions identical (or very similar) to this one, that you submitted and then deleted)

Anyway, here's a simple example for demo purposes.

dll00.c :

#include <stdio.h>
#include <stdlib.h>

#if defined(_WIN32)
#  define DLL00_EXPORT_API __declspec(dllexport)
#else
#  define DLL00_EXPORT_API
#endif


#if defined(__cplusplus)
extern "C" {
#endif

DLL00_EXPORT_API double **init(double **ppMat, int m, int n);
DLL00_EXPORT_API int cleanup(double **ppMat, int m);

#if defined(__cplusplus)
}
#endif

DLL00_EXPORT_API double **init(double **ppMat, int m, int n) {
    const double factor = 7.0;
    printf("\n----- FROM C: Multiplying input matrix by: %.3f\n", factor);
    double **ret = malloc(m * sizeof(double*));
    for (int i = 0; i < m; i++) {
        ret[i] = malloc(n * sizeof(double));
        for (int j = 0; j < n; j++) {
            ret[i][j] = ppMat[i][j] * factor;
        }
    }
    return ret;
}

DLL00_EXPORT_API int cleanup(double **ppMat, int m) {
    int ret = 0;
    if (ppMat) {
        printf("\n----- FROM C: free\n");
        for (int i = 0; i < m; i++) {
            free(ppMat[i]);
            ret++;
            ppMat[i] = NULL;
        }
        free(ppMat);
    }
    return ++ret;
}

code00.py :

#!/usr/bin/env python3

import sys
import ctypes as ct
from pprint import pprint as pp


DLL_NAME = "./dll00.dll"


def ptr2d_to_mat(ptr, rows, cols):
    return tuple(tuple(ptr[i][j] for j in range(cols)) for i in range(rows))


def main():
    dll00 = ct.CDLL(DLL_NAME)
    init = dll00.init
    cleanup = dll00.cleanup

    rows = 4
    cols = 6

    DblPtr = ct.POINTER(ct.c_double)
    DblPtrPtr = ct.POINTER(DblPtr)

    init.argtypes = [DblPtrPtr, ct.c_int, ct.c_int]
    init.restype = DblPtrPtr
    cleanup.argtypes = [DblPtrPtr, ct.c_int]
    cleanup.restype = ct.c_int

    DblPtrArr = DblPtr * rows

    DblArr = ct.c_double * cols
    DblArrArr = DblArr * rows

    first_value = 6
    in_mat = tuple(tuple(range(cols * i + first_value, cols * (i + 1) + first_value)) for i in range(rows))
    print("Input matrix:")
    pp(in_mat)
    in_arr = DblArrArr(*in_mat)
    in_ptr = ct.cast(DblPtrArr(*(ct.cast(row, DblPtr) for row in in_arr)), DblPtrPtr)  # Cast each row and the final array to (corresponding) pointers
    out_ptr = init(in_ptr, rows, cols)
    out_mat = ptr2d_to_mat(out_ptr, rows, cols)
    cleanup(out_ptr, rows)
    print("\nOutput matrix:")
    pp(out_mat)


if __name__ == "__main__":
    print("Python {0:s} {1:d}bit on {2:s}\n".format(" ".join(item.strip() for item in sys.version.split("\n")), 64 if sys.maxsize > 0x100000000 else 32, sys.platform))
    main()
    print("\nDone.")

Output :

 [cfati@CFATI-5510-0:e:\Work\Dev\StackOverflow\q058226790]> sopr.bat *** Set shorter prompt to better fit when pasted in StackOverflow (or other) pages *** [prompt]> "c:\Install\x86\Microsoft\Visual Studio Community\2017\VC\Auxiliary\Build\vcvarsall.bat" x64 ********************************************************************** ** Visual Studio 2017 Developer Command Prompt v15.9.16 ** Copyright (c) 2017 Microsoft Corporation ********************************************************************** [vcvarsall.bat] Environment initialized for: 'x64' [prompt]> dir /b code00.py dll00.c [prompt]> cl /nologo /DDLL dll00.c /link /NOLOGO /DLL /OUT:dll00.dll dll00.c Creating library dll00.lib and object dll00.exp [prompt]> dir /b code00.py dll00.c dll00.dll dll00.exp dll00.lib dll00.obj [prompt]> "e:\Work\Dev\VEnvs\py_064_03.07.03_test0\Scripts\python.exe" code00.py Python 3.7.3 (v3.7.3:ef4ec6ed12, Mar 25 2019, 22:22:05) [MSC v.1916 64 bit (AMD64)] 64bit on win32 Input matrix: ((6, 7, 8, 9, 10, 11), (12, 13, 14, 15, 16, 17), (18, 19, 20, 21, 22, 23), (24, 25, 26, 27, 28, 29)) ----- FROM C: Multiplying input matrix by: 7.000 ----- FROM C: free Output matrix: ((42.0, 49.0, 56.0, 63.0, 70.0, 77.0), (84.0, 91.0, 98.0, 105.0, 112.0, 119.0), (126.0, 133.0, 140.0, 147.0, 154.0, 161.0), (168.0, 175.0, 182.0, 189.0, 196.0, 203.0)) Done.

You can also take a look at [SO]: Problems with passing and getting arrays for a C function using ctypes (@CristiFati's answer) , which is very similar (almost identical, I'd say) to this one.