non-blocking lock with 'with' statement

Question

As far as I know, the following code will be blocked if lock is already acquired by another thread.

It seems that non-blocking can be implemented by lock.acquire(0) , but instead I have to use try-finally block instead with block.

lock = threading.Lock()

def func():
 with lock:
  # do something...

Is there any method to implement non-blocking lock acquisition?

Answer 1

@contextmanager
def nonblocking(lock):
    locked = lock.acquire(False)
    try:
        yield locked
    finally:
        if locked:
            lock.release()

lock = threading.Lock()
with nonblocking(lock) as locked:
    if locked:
        do_stuff()

Answer 2

Is there any method to implement non-blocking lock acquisition?

Yes. Just raise an exception if the lock can't be acquired immediately. Something like:

@contextlib.contextmanager
def non_blocking_lock(lock=threading.Lock()):
    if not lock.acquire(blocking=False):
        raise WouldBlockError
    try:
        yield lock
    finally:
        lock.release()

Usage:

with non_blocking_lock():
    # run with the lock acquired

Answer 3

You can implement your own lock-like object that behaves as you wish:

class MyLock(object):
    def __init__(self):
        self._lock = threading.Lock()
        self._locked = False
    def __enter__(self):
        locked = self._lock.acquire(False)
        self._locked = locked
        return locked
    def __exit__(self, *args):
        if self._locked:
            self._lock.release()
            self._locked = False

I tested it with this code:

In [5]: ml = MyLock()
In [6]: with ml as aq:
   ...:     print "got it?!", aq
   ...:     with ml as try2:
   ...:         print "got it second time???", try2

Output is:

got it?! True
got it second time??? False

You can also implement lock and unlock methods that use the inner lock, but you get the picture, fit it to your needs.

Answer 4

If you need a context manager that acquires a lock in a non-blocking manner, but still retries until the lock can finally be acquired, you could do like this:

@contextlib.contextmanager
def non_blocking_lock(lock : threading.Lock):
    # Waits as long as the lock can not be acquired, but releases the GIL in the meanwhile
    while not lock.acquire(blocking=False):
        pass

    try:
        yield   # Lock has been successfully acquired
    finally:
        lock.release()

It can be used exactly like the normal lock context manager:

class TestClass:
    def __init__(self):
         self._lock = threading.Lock()
    
    def method(self):
         with non_blocking_lock(self._lock):
         # do something that should be only done from one thread at once

... with the difference, that the lock is non-blocking and doesn't hold the GIL until the lock is released. I used it to fix some deadlocks.

The difference to the other solutions is, that the code eventually gets executed and the context manager does not simply return a false or throw an exception when the lock couldn't be acquired.

Correct me if you see any caveats with this solution.

Answer 5

The whole point of a lock is to ensure that certain sections of your program will only be executed by one thread or process at a time. This is achieved by blocking any threads/processes trying to acquire the lock while something else holds it.

If you don't want acquiring the lock to block, why are you using a lock in the first place? Presumably so that you can do something else while you wait?

To attempt to acquire the a lock l without blocking, call l.acquire(blocking=False) . This will immediately return False if the lock was not acquired. If the lock was acquired it returns True , and the you will continue to hold the lock until you call its release() method.

This form, however, isn't particularly useful with the with statement. Usually you want the controlled code (the indented suite after the with ) to run only when the lock has been acquired. not to query whether it has or not and take two alternative actions.

Answer 6

If you want to use the with statement with a non-blocking lock, you could also first check if it is locked. If it is, then you don't go into the with block. Eg:

lock = threading.Lock()

def func():
    if not lock.locked():
        with lock:
            # do something...