Keep process running while program continues C++

Question

I want to be able to call a function in a python script through C++. The function will print some text to stdout.

However, the python script takes about 1 minute to load the global variables to memory, so I want to keep the process alive in interactive mode, where I would be able to simply call the function again without loading all the variables again.

I know that I can run system commands using system , but the command must terminate before I can use the output. I am also aware that I can use CPython and call PyRun_SimpleFileEx on the file, but this will have to reload all the variables again

I don't have any code to show since I don't know how to do this. Is there any way I can do this without rewriting the whole python script in C++?

Answer 1

Every PyRun_* function runs in the __main__ module, so any functions or variables you define stick around for future executions.

For example, say we want to run the do_it function from this file (helper.py) multiple times:

from datetime import datetime as dt
from time import sleep
print("First load")

def do_it():
    print("Helper function called on demand: %s" % dt.now())
    sleep(1)

do_it()

We can simply invoke PyRun_SimpleFile once, and then access the function again using PyRun_SimpleString("do_it()\\n"); .

#define PY_SSIZE_T_CLEAN
#include <Python.h>

int main(int argc, char *argv[])
{
    wchar_t *program = Py_DecodeLocale(argv[0], NULL);
    if (program == NULL) {
        fprintf(stderr, "Fatal error: cannot decode argv[0]\n");
        exit(1);
    }
    Py_SetProgramName(program);  /* optional but recommended */
    Py_Initialize();
    FILE *fp = fopen("helper.py", "rb");
    PyRun_SimpleFile(fp, "helper.py");
    for (int i = 0; i < 10; i++) {
        PyRun_SimpleString("do_it()\n");
    }
    if (Py_FinalizeEx() < 0) {
        exit(120);
    }
    PyMem_RawFree(program);
    return 0;
}

Note that this makes use of the "Very High Level Embedding" and is thus limited to evaluating strings. For more complex arguments you may want to use the "Pure embedding" instead to grab the PyFunctionObject from the module and invoke it.

Finally, you may want to look at pybind11 , a C++11 bridge between Python and C++. This should make it very easy to grab Python the function and call it directly from C++.