简体   繁体   中英

Data transfer between C++ and Python

I would like to share memory between C++ and Python.

My problem:

  1. I am working with big data sets (up to 6 GB of RAM) in C++. All calculations are done in c++.
  2. Then, I want to "paste" all my results to a Python program. But I can only write my data on disk, and then read that file from Python, which is not efficient.

Is there any way to "map" memory corresponding to C++ variables so that I may access the data from Python? I don't want to copy 6GB of data onto a hard drive.

First path : I think the more appropriate way for you to go is ctypes . You can create a shared library, and then load the functions of the shared library in Python, and fill all the data containers you want in Python.

In Windows, you can create a DLL, and in Linux you can create a shared .so library.

Now this has the advantage that this will be independent of your Python version.

Second path : I think it's less appropriate but you can get it to work, which is the Python C Extension . With this, you can call Python data containers ( PyObject s) and fill them inside C.

However, the code you compile here will always need to be linked to Python libraries.

Which one to use? :

  • Use ctypes if you have some functions you want to call in C/C++, and then do the rest of the work in Python.
  • Use Python C Extension if you have some functions you want to call in Python, and you want to do the rest in C/C++.

With both options, you can transfer huge blocks of memory between C++ and Python without necessarily involving any disk read/write operations.

Hope this helps.

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM