Python Thread not releasing lock

Question

While creating a simple consumer-producer thread structure using locks in Python I encountered some issues which caused the program to create unexpected output.

import threading
from time import sleep
import random

class AI:
    def __init__(self):
        self.a = None
        self.mainLock = threading.Lock()
        threading.Thread(target=self.producer,daemon=True).start()
        threading.Thread(target=self.consumer,daemon=True).start()

    def producer(self):
        while True:
            self.mainLock.acquire()
            sleep(1)
            temp = random.randint(-1,10)
            print(f"Produced {temp}")
            self.a = temp
            self.mainLock.release()

    def consumer(self):
        while True:
            self.mainLock.acquire()
            if(self.a and self.a>0):
                sleep(1.5)
                print(f"Consumed {self.a}")
                self.a = None
            self.mainLock.release()

a = AI()
input()

Output -

Produced 0
Produced 8
Produced 9
Produced 1
Produced 9
Produced 10
Produced 5
Produced 1

Which is clearly not what was expected here. The expected output would have contained consumer after producer. But if I add any statement after releasing the lock inside the producer then the code runs fine as it was expected to run.

Code -

import threading
from time import sleep
import random

class AI:
    def __init__(self):
        self.a = None
        self.mainLock = threading.Lock()
        threading.Thread(target=self.producer,daemon=True).start()
        threading.Thread(target=self.consumer,daemon=True).start()

    def producer(self):
        while True:
            self.mainLock.acquire()
            sleep(1)
            temp = random.randint(-1,10)
            print(f"Produced {temp}")
            self.a = temp
            self.mainLock.release()
            ## = Newly added line
            ########################
            print("released")
            ########################

    def consumer(self):
        while True:
            self.mainLock.acquire()
            if(self.a and self.a>0):
                sleep(1.5)
                print(f"Consumed {self.a}")
                self.a = None
            self.mainLock.release()

a = AI()
input()

Output -

Produced 10
released
Consumed 10
Produced 7
released
Consumed 7
Produced 2
released
Consumed 2

Even if I replace the print statement with a sleep statement then also the code works irrespective of how small the sleep duration is it still works fine.

Example -

sleep(0.0000000000000000000000000000000000000000000000000000000000000000000000001)

Why is it happening? And how was the code able the jump back from consumer to producer without any print or sleep after consumer's release without which the code was not able to jump from producer to consumer?

Answer 1

Releasing a lock does not automatically ensure that some other waiting thread will immediately acquire the lock.

This, in particular, is an anti-pattern:

 while (True):
     someLock.acquire()
     ...
     someLock.relase()

The problem is, the very next thing that the thread does after releasing the lock is, it acquires it again.

Imagine you are in the bathroom with the door locked. Somebody else is outside, waiting to get in. You've made them wait a good long while, so maybe they sat down. Suddenly, you open the door, you step out, and before the other person even has time to stand up, you step back inside and lock the door again.

That's roughly what's happening in your program.

Locks are good for one thing, and for one thing only: You can use them to stop two threads from using the same resources at the same time. If you try to use them for any other purpose (eg, to control the order in which threads do something), you're going to have a hard time.

---

In many programming languages/libraries, the most primitive means for one thread to signal another thread is called a condition variable . See https://docs.python.org/3/library/threading.html#condition-objects

Answer 2

I don't think you have the right in either code example to expect any particular output because the results you will get will depend on how and when the threads are given control of the processor that they are both sharing. What you have is a race condition , defined as follows:

A race condition arises in software when a computer program, to operate properly, depends on the sequence or timing of the program's processes or threads.

There are points in time where neither thread holds the lock, which is the case obviously when both threads start. Looking at producer , if it is the first to run, it acquires the lock, prints its message and then releases its lock. But surely it is possible to immediately return to the top of the loop and re-acquire the lock again. Meanwhile the main thread may eventually get dispatched and will terminate and then both the producer and consumer threads being daemon threads will terminate too without perhaps the consumer thread ever getting a chance to rum (which is what you have observed). Any change to the producer thread such as an I/O statement (ie print ) or sleep , might however cause it to go into a wait state and give the consumer thread a chance to run (which you have also observed). But even in this case, the main thread may be dispatched and terminate before the consumer thread can print anything.

Answer 3

Releasing a mutex by one thread does not mean that another thread will necessarily acquire it, in this case it can again be acquired by the first thread. This order is not deterministic for most operating systems. How often the second thread will be able to acquire mutex already depends on the scheduling of the OS threads.

For example, on Windows your example, in general, works (Produced, Consumed interleaved). On Linux (I will assume that you use it) the second thread is starving. It remains to either add commands after release , like sleep or use other approaches to implement the producer-consumer pattern, for example, a queue .