Threading in 'while True' loop

Question

First of all, I've only started working with Python a month ago, so I don't have deep knowledge about anything.

In a project I'm trying to collect the results of multiple (simultaneous) functions in a database, infinitely until I tell it to stop. In an earlier attempt, I successfully used multiprocessing to do what I need, but since I now need to collect all the results of those functions for a database within the main, I switched to threading instead.

Basically, what I'm trying to do is:

collect1 = Thread(target=collect_data1)
collect2 = Thread(target=collect_data2)
send1 = Thread(target=send_data1)
send2 = Thread(target=send_data2)
collect = (collect1, collect2)
send = (send1, send2)
while True:
    try:
        for thread in collect:
            thread.start()
        for thread in collect:
            thread.join()
        for thread in send:
            thread.start()
        for thread in send:
            thread.join()
    except KeyboardInterrupt:
        break

Now, obviously I can't just restart the threads. Nor explicitly kill them. The functions within the threads can theoretically be stopped at any point, so terminate() from multiprocessing was fine.

I was thinking whether something like the following could work (at least PyCharm is fine with it, so it seems to work) or if it creates a memory leak (which I assume), because the threads are never properly closed or deleted, at least as far as I can tell from researching. Again, I'm new to Python, so I don't know anything about this aspect.

Code:

while True:
        try:
            collect1 = Thread(target=collect_data1)
            collect2 = Thread(target=collect_data2)
            send1 = Thread(target=send_data1)
            send2 = Thread(target=send_data2)
            collect = (collect1, collect2)
            send = (send1, send2)
            for thread in collect:
                thread.start()
            for thread in collect:
                thread.join()
            for thread in send:
                thread.start()
            for thread in send:
                thread.join()
        except KeyboardInterrupt:
            break

I feel like this approach seems too good to be true, especially since I've never came across a similar solution during my research.

Anyways, any input is appreciated.

Have a good day.

Answer 1

You explicitly await the threads' termination in loops containing thread.join() , so no memory leak takes place, and your code is fine. If you're worried that you don't dispose of the thread objects in any way after they terminate, it will be done automatically as soon as they aren't used anymore, so this souldn't be a concern neither.