如何为在 C++ 应用程序中使用 embed python 创建的新线程状态获取 GIL

Question

I have a C++ application where we provide a python editor(which uses python 3.7), users can input and execute the python code in the editor. Following is the sample code where we are creating a new thread state and making it current before executing the script provided by users.

#include <python.h>

PyThreadState* globalthread;

void executescript()
{
    PyEval_SaveThread();
    PyThreadState* ts = PyThreadState_New(globalthread->interp);
    int gilstate = PyGILState_Check(); //check whether GIL is with current thread or not
    if (!gilstate)
    {
        PyEval_RestoreThread(ts);
    }
    else
    {
        
        PyThreadState_Swap(ts);
    }
    // get the thread state with which GIL is currently assigned
    PyThreadState * gilthreadstate = PyGILState_GetThisThreadState(); 
    //----> the above line return pointer to global thread state (gilthreadstate == globalthread)
    //----> which means the GIL is not acquired by the new current thread
    gilstate = PyGILState_Check(); //results in zero as GIL is not with current thread
    std::string str = "def script():\n\timport sys\n\tsys.path.append('C:\\Python\\Python37\\Lib\\site-packages')\n\tprint(sys.path)\n\timport numpy\n\tarr = numpy.array([1, 2, 3, 4, 5])\n\tprint(arr)\nscript()";
    PyRun_SimpleString(str.c_str());
    PyThreadState_Clear(ts);
    PyThreadState_DeleteCurrent();
    PyEval_RestoreThread(globalthread);
    //following stmt returns 1 as the current thread is global thread and it has GIL with it
    gilstate = PyGILState_Check(); 
}

int main()
{
    Py_Initialize();
    PyEval_InitThreads();
    globalthread = PyThreadState_Get();
    executescript();
    PyThreadState_Swap(globalthread);
    Py_FinalizeEx();
    return 0;
} 

The new current thread is not acquiring the GIL when PyEval_RestoreThread or PyThreadState_Swap is called on it. Python scripts having import numpy or import pandas will end in deadlock due to this. Can someone let me know how to acquire GIL for the new thread state or where the code is wrong?

Answer 1

Comment by hn-1-8e9-wheres-my-share-m has resolved the issue by creating a new C++ thread. Here is my modified code.

#include <python.h>
#include <thread>
PyThreadState* globalthread;

void execute()
{
    PyThreadState* ts = PyThreadState_New(globalthread->interp);

    int gilstate = PyGILState_Check();
    if (!gilstate)
    {
        PyEval_RestoreThread(ts);
    }
    else
    {

        PyThreadState_Swap(ts);
    }
    std::string str = "def script():\n\timport sys\n\tsys.path.append('C:\\Python\\Python37\\Lib\\site-packages')\n\tprint(sys.path)\n\timport numpy\n\tarr = numpy.array([1, 2, 3, 4, 5])\n\tprint(arr)\nscript()";
    PyRun_SimpleString(str.c_str());
    PyThreadState_Clear(ts);
    PyThreadState_DeleteCurrent();
}

int main()
{
    Py_Initialize();
    globalthread = PyThreadState_Get();
    PyEval_SaveThread();
    std::thread t(executePythonScript);
    t.join();
    PyEval_RestoreThread(globalthread);
    //PyThreadState_Swap(globalthread);
    Py_FinalizeEx();
    return 0;
}

Thank you for pointing out the post.