C ++和Python之间的数据传输

Question

I would like to share memory between C++ and Python.

My problem:

1. I am working with big data sets (up to 6 GB of RAM) in C++. All calculations are done in c++.
2. Then, I want to "paste" all my results to a Python program. But I can only write my data on disk, and then read that file from Python, which is not efficient.

Is there any way to "map" memory corresponding to C++ variables so that I may access the data from Python? I don't want to copy 6GB of data onto a hard drive.

Answer 1

First path : I think the more appropriate way for you to go is ctypes . You can create a shared library, and then load the functions of the shared library in Python, and fill all the data containers you want in Python.

In Windows, you can create a DLL, and in Linux you can create a shared .so library.

Now this has the advantage that this will be independent of your Python version.

Second path : I think it's less appropriate but you can get it to work, which is the Python C Extension . With this, you can call Python data containers ( PyObject s) and fill them inside C.

However, the code you compile here will always need to be linked to Python libraries.

Which one to use? :

• Use ctypes if you have some functions you want to call in C/C++, and then do the rest of the work in Python.
• Use Python C Extension if you have some functions you want to call in Python, and you want to do the rest in C/C++.

With both options, you can transfer huge blocks of memory between C++ and Python without necessarily involving any disk read/write operations.

Hope this helps.