Different behaviour when using context class and contextmananger method decorator in Python

Question

I get different behaviour in the following two scenarios when trying to use Python (3.5)'s context manager. I'm trying to handle KeyboardInterrupt exceptions to my threaded program gracefully by using a proper shutdown procedure in combination with a context manager, but it seems like I can't get this to work in the second case and I can't see why not.

Common to both cases is a generic "job" task that uses threading:

import threading
class Job(threading.Thread):
    def run(self):
        self.active = True

        while self.active:
            continue

    def stop(self):
        self.active = False

Once started with start (a method provided by the threading.Thread parent class, which calls run internally), it can be stopped by calling stop .

The first way I tried to do this was to use the built-in __enter__ and __exit__ methods so as to take advantage of Python's with statement support:

class Context(object):
    def __init__(self):
        self.active = False

    def __enter__(self):
        print("Entering context")
        self.job = Job()
        self.job.start()
        return self.job

    def __exit__(self, exc_type, exc_value, traceback):
        print("Exiting context")
        self.job.stop()
        self.job.join()
        print("Job stopped")

I run it using the following code:

with Context():
    while input() != "stop":
        continue

This waits until the user types "stop" and presses enter. If during this loop the user instead presses Ctrl+C to create a KeyboardInterrupt , the __exit__ method is still called:

Entering context
^CExiting context
Job stopped
Traceback (most recent call last):
  File "tmp2.py", line 48, in <module>
    while input() != "stop":
KeyboardInterrupt

The second way I tried to do this was to create a function using the @contextmanager decorator:

from contextlib import contextmanager

@contextmanager
def job_context():
    print("Entering context")
    job = Job()
    job.start()
    yield job
    print("Exiting context")
    job.stop()
    job.join()
    print("Job stopped")

I again run it using the with statement:

with job_context():
    while input() != "stop":
        continue

But when I run it, and press Ctrl+C , the code after the yield - the equivalent of the __exit__ method in the first example, is not executed. Instead, the Python script continues to run in the infinite loop. To stop the program I have to press Ctrl+C a second time, at which point the code after yield is not executed:

Entering context
^CTraceback (most recent call last):
  File "tmp2.py", line 42, in <module>
    while input() != "stop":
KeyboardInterrupt
^CException ignored in: <module 'threading' from '/usr/lib/python3.5/threading.py'>
Traceback (most recent call last):
  File "/usr/lib/python3.5/threading.py", line 1288, in _shutdown
    t.join()
  File "/usr/lib/python3.5/threading.py", line 1054, in join
    self._wait_for_tstate_lock()
  File "/usr/lib/python3.5/threading.py", line 1070, in _wait_for_tstate_lock
    elif lock.acquire(block, timeout):
KeyboardInterrupt

You can see the ^C symbols where I pressed Ctrl+C to create the interrupts. What's different about the second case that it doesn't perform the shutdown code equivalent to __exit__ in the first case?

Answer 1

Per the documentation :

If an unhandled exception occurs in the block, it is reraised inside the generator at the point where the yield occurred. Thus, you can use a try ... except ... finally statement to trap the error (if any), or ensure that some cleanup takes place.

In your case, this would look like:

@contextmanager
def job_context():
    print("Entering context")
    job = Job()
    job.start()
    try:
        yield job
    finally:
        print("Exiting context")
        job.stop()
        job.join()
        print("Job stopped")