Boost python producer consumer problem (execute boost python thread, std::mutex deadlock)

Question

I'm trying to create boost python module and test producer & consumer problem.

But I faced some error and problems...

Below image shows my goal architecture.

I implemented boost python module

and here is python code snippet (main.py)

import threading
import datetime

import PyDataTest

queue = None


def thread_function():
    print('[thread_function] start...')
    while True:
        print('[thread_function] try Pop')
        data = queue.Pop()

        print(f'[thread_function] Pop: {data}')


def wait_thread_function():
    print('[wait_thread_function] start...')

    wait_thread = threading.Thread(target=thread_function)
    wait_thread.start()

    wait_thread.join()


def gen_thread_function():
    gen_thread_obj = PyDataTest.CustomThread()

    gen_thread = threading.Thread(target=gen_thread_obj.Execute)
    gen_thread.start()
    gen_thread.join()


if __name__ == '__main__':
    print('main start')
    queue = PyDataTest.GetQueueInstance()

    wait_thread_func = threading.Thread(target=wait_thread_function, args=())
    wait_thread_func.start()

    gen_thread_func = threading.Thread(target=gen_thread_function)
    gen_thread_func.start()

    timeVal = datetime.datetime.now()

    while True:
        currentTime = datetime.datetime.now()

        if (currentTime - timeVal).seconds > 10:
            print('Python Main running ...')

    thread.join()
    gen_thread.join()

I faced two problems

1. When "wait_thread_function" start first, python program was blocked
2. When "gen_thread_function" start first, showed Abort error message

So my question is

1. I guess No.1 problem is cause by python GIL, because "IntegerQueue" Pop function using std::mutex so I want know "how to release GIL inside Pop function"
2. How to execute std::thread in boost python from python file?
3. Is it possible IntegerQueue shared python file and boost python module?
4. If queue stored custom class type (not primitive datatype) it also shared with python file and boost python module?

My final goal is based on producer & consumer model and custom object type data shared via Queue object

Please help me

Thanks

Environment

• python: 3.9.2 (AMD64)
• C++: Visual Studio 2019
• Boost: 1.75.0

Additional information

Pop function code snippet

    bool Pop(T& obj)
    {
        std::unique_lock<std::shared_mutex> ul(sm);

        cv.wait(ul, [this] {return (!con.empty() || !isRun); });

        if (!isRun)
        {
            return false;
        }

        if (!con.empty())
        {
            obj = con.front();
            con.pop();
            return true;
        }

        return false;
    }

========================================================================

Error message when "gen_thread_function" running first

Answer 1

Ok. I found how to solve that problems..

1. I found the way using scoped gil release. this question very helpful for me

2. When I run the thread, I face error windows above... it cause by I cannot join the thread in C++ module. I added thread join code inside run function the error goes away

3. I thought directly sharing a queue has lots of problems.. so I changed architecture. Queue object resident inside C++ module and expose the function for accessing queue. And it works nicely

4. It is same as solution 3, I solve the problem exposing C++ class

Thanks