[英]Python timeout context manager with threads
I have timeout
context manager that works perfectly with signals but it raises error in multithread mode because signals work only in main thread.我有timeout
上下文管理器,它与信号完美配合,但它在多线程模式下引发错误,因为信号只在主线程中工作。
def timeout_handler(signum, frame):
raise TimeoutException()
@contextmanager
def timeout(seconds):
old_handler = signal.signal(signal.SIGALRM, timeout_handler)
signal.alarm(seconds)
try:
yield
finally:
signal.alarm(0)
signal.signal(signal.SIGALRM, old_handler)
I've seen decorator implementation of timeout
but I don't know how to pass yield
inside class derived from threading.Thread
.我已经看到了timeout
的装饰器实现,但我不知道如何在从threading.Thread
派生的类中传递yield
。 My variant won't work.我的变种不起作用。
@contextmanager
def timelimit(seconds):
class FuncThread(threading.Thread):
def run(self):
yield
it = FuncThread()
it.start()
it.join(seconds)
if it.isAlive():
raise TimeoutException()
If the code guarded by the context manager is loop-based, consider handling this the way people handle thread killing. 如果上下文管理器保护的代码是基于循环的,请考虑以人们处理线程查杀的方式处理它。 Killing another thread is generally unsafe, so the standard approach is to have the controlling thread set a flag that's visible to the worker thread. 杀死另一个线程通常是不安全的,因此标准方法是让控制线程设置一个工作线程可见的标志。 The worker thread periodically checks that flag and cleanly shuts itself down. 工作线程定期检查该标志并干净地关闭自己。 Here's how you can do something analogous with timeouts: 以下是您可以执行与超时类似的操作:
class timeout(object):
def __init__(self, seconds):
self.seconds = seconds
def __enter__(self):
self.die_after = time.time() + self.seconds
return self
def __exit__(self, type, value, traceback):
pass
@property
def timed_out(self):
return time.time() > self.die_after
Here's a single-threaded usage example: 这是一个单线程用法示例:
with timeout(1) as t:
while True: # this will take a long time without a timeout
# periodically check for timeouts
if t.timed_out:
break # or raise an exception
# do some "useful" work
print "."
time.sleep(0.2)
and a multithreaded one: 和一个多线程的:
import thread
def print_for_n_secs(string, seconds):
with timeout(seconds) as t:
while True:
if t.timed_out:
break # or raise an exception
print string,
time.sleep(0.5)
for i in xrange(5):
thread.start_new_thread(print_for_n_secs,
('thread%d' % (i,), 2))
time.sleep(0.25)
This approach is more intrusive than using signals but it works for arbitrary threads. 这种方法比使用信号更具侵入性,但它适用于任意线程。
I cannot see a way of doing what you are proposing with a context manager, you cannot yield
the flow from one thread to another. 我无法通过上下文管理器看到你提出的方法,你不能从一个线程到另一个线程yield
流量。 What I would do is wrap your function with an interrutable thread with the timeout. 我要做的是用一个带有超时的可混合线程包装你的函数。 Here is a recipe for that. 这是一个食谱 。
You will have an extra thread and the syntax won't be as nice but it would work. 你将有一个额外的线程,语法不会很好,但它会工作。
I know it's late but I'm only just reading this, but what about creating your own signaller/context manager? 我知道现在已经很晚了但我只是在读这个,但是创建自己的信号器/上下文管理器怎么样? I'm new to python would love feedback from experienced devs this implementation. 我是python的新手,会喜欢经验丰富的开发人员的反馈。
This is based off of the answer from "Mr Fooz" 这是基于“Fooz先生”的答案
class TimeoutSignaller(Thread):
def __init__(self, limit, handler):
Thread.__init__(self)
self.limit = limit
self.running = True
self.handler = handler
assert callable(handler), "Timeout Handler needs to be a method"
def run(self):
timeout_limit = datetime.datetime.now() + datetime.timedelta(seconds=self.limit)
while self.running:
if datetime.datetime.now() >= timeout_limit:
self.handler()
self.stop_run()
break
def stop_run(self):
self.running = False
class ProcessContextManager:
def __init__(self, process, seconds=0, minutes=0, hours=0):
self.seconds = (hours * 3600) + (minutes * 60) + seconds
self.process = process
self.signal = TimeoutSignaller(self.seconds, self.signal_handler)
def __enter__(self):
self.signal.start()
return self.process
def __exit__(self, exc_type, exc_val, exc_tb):
self.signal.stop_run()
def signal_handler(self):
# Make process terminate however you like
# using self.process reference
raise TimeoutError("Process took too long to execute")
Use case: 使用案例:
with ProcessContextManager(my_proc) as p:
# do stuff e.g.
p.execute()
Similar implementation as Mr Fooz but using the contextlib
library:与 Mr Fooz 类似的实现,但使用contextlib
库:
import time
from contextlib import contextmanager
@contextmanager
def timeout(seconds):
"""
A simple context manager to enable timeouts.
Example:
with timeout(5) as t:
while True:
if t():
# handle
"""
stop = time.time() + seconds
def timed_out():
return time.time() > stop
yield timed_out
Timeouts for system calls are done with signals. 系统调用超时由信号完成。 Most blocking system calls return with EINTR when a signal happens, so you can use alarm to implement timeouts. 当信号发生时,大多数阻塞系统调用都会返回EINTR,因此您可以使用警报来实现超时。
Here's a context manager that works with most system calls, causing IOError to be raised from a blocking system call if it takes too long. 这是一个与大多数系统调用一起使用的上下文管理器,如果花费的时间太长,会导致阻塞系统调用引发IOError。
import signal, errno
from contextlib import contextmanager
import fcntl
@contextmanager
def timeout(seconds):
def timeout_handler(signum, frame):
pass
original_handler = signal.signal(signal.SIGALRM, timeout_handler)
try:
signal.alarm(seconds)
yield
finally:
signal.alarm(0)
signal.signal(signal.SIGALRM, original_handler)
with timeout(1):
f = open("test.lck", "w")
try:
fcntl.flock(f.fileno(), fcntl.LOCK_EX)
except IOError, e:
if e.errno != errno.EINTR:
raise e
print "Lock timed out"
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.