如何在 Python 中限制 function 执行的最有效方法

Question

我有一个 function，它会定期做一些事情。 该时间段设置为一分钟，因此我需要以某种方式限制执行时间。 我曾尝试使用multiprocessing ，但它大大增加了执行时间（从 <1 秒到 2 - 10 秒）。

有没有更好的方法来配置最大执行时间（下面代码片段中的example_function ）？

我试过使用signal ，但是它不能很好地与调度一起使用。

我在这个项目中使用 Python 3.9 （计划更新到3.10 ）

示例代码：

from apscheduler.schedulers.blocking import BlockingScheduler
from multiprocessing import Process


def example_function():
  ...
  here is some processing
  ...


def scheduled_function():
  limit = 4
  p = Process(target=example_function, name='Example Function')

  p.start()
  p.join(timeout=limit)
  p.terminate()


scheduler = BlockingScheduler()
scheduler.add_job(scheduled_function, 'cron', minute='*/1')
scheduler.start()

谢谢

---

编辑

我找到了使用threading而不是multiprocessing的例子。 似乎执行时间更好，只是需要一种不同的方法来处理超时/成功结果

from apscheduler.schedulers.blocking import BlockingScheduler
from threading import Thread


finished = False


def example_function():
  ...
  here is some processing
  ...
  global finished
  finished = True


def scheduled_function():
  limit = 4
  p = Thread(target=example_function)

  p.start()
  p.join(timeout=limit)

  global finished
  if finished:
    return "Finished OK"
  else:
    return "Timeout"


scheduler = BlockingScheduler()
scheduler.add_job(scheduled_function, 'cron', minute='*/1')
scheduler.start()

Answer 1

我试过使用信号，但是它不能很好地与调度一起工作。

我通常使用信号超时，但由于它不适用于 BlockingScheduler（ ValueError: signal only works in main thread ），这是每 x 秒触发一次scheduled_function的解决方法：

def example_function(i, limit=4):
    with timeout(limit):
        try:
            ...
        except TimeoutError:
            ...

def scheduled_function(cron):
    with Timer() as t:
        with timeout(cron - 1):
            with Pool(1) as p:
                p.map(partial(example_function, limit=4), range(1))
    # sleep for remaining time
    time.sleep(cron - t.secs)

while True:
   scheduled_function(cron=10)  # every 10s

这是完整的代码：

import random
import time
import signal
from functools import partial
from contextlib import contextmanager
from multiprocessing import Pool
from datetime import datetime


class Timer:
    def __enter__(self):
        self.start_time = time.perf_counter()
        return self

    def __exit__(self, exc_type, exc_val, exc_tb):
        end_time = time.perf_counter()
        self.secs = end_time - self.start_time  # seconds


@contextmanager
def timeout(t):
    """https://www.jujens.eu/posts/en/2018/Jun/02/python-timeout-function/"""
    signal.signal(signal.SIGALRM, raise_timeout)
    signal.alarm(t)

    try:
        yield
    except TimeoutError:
        pass
    finally:
        signal.signal(signal.SIGALRM, signal.SIG_IGN)


def raise_timeout(signum, frame):
    raise TimeoutError


def example_function(i, limit=4):
    with timeout(limit):
        try:
            a = random.uniform(1, 5)
            print(f"running task {i} for {a} seconds")
            time.sleep(a)
            print(f"[OK] task {i} finished")
        except TimeoutError:
            print(f"[NOT OK] task {i} did not finish: {a}>{limit}")

    print(f"timeout={limit}s has expired -> exiting task {i}")


def scheduled_function(cron):
    print("starting scheduled_function at", datetime.now().strftime("%Mmin%Ss"))
    with Timer() as t:
        with timeout(cron - 1):
            print("running small 4-secs jobs")
            with Pool(1) as p:
                p.map(partial(example_function, limit=4), range(1))
    time.sleep(cron - t.secs)
    print("1min -> exiting")


def main():
    while True:
        scheduled_function(cron=10)  # every 10s


if __name__ == "__main__":
    main()

和 output：

starting scheduled_function at '10s'                   <-- starting every 10s
running small 4-secs jobs
running task 0 for 3.0037614230572447 seconds
[OK] task 0 finished
timeout=4s has expired -> exiting task 0                 <-- inner task timeout=4s
10s -> exiting                                           <-- outer job timeout=10s

starting scheduled_function at '20s'                   <-- starting every 10s
running small 4-secs jobs
running task 0 for 4.198487250169565 seconds
[NOT OK] task 0 did not finish: 4.198487250169565>4       <-- caught timeout error
timeout=4s has expired -> exiting task 0

starting scheduled_function at '30s'
running small 4-secs jobs
running task 0 for 3.489094988621927 seconds
[OK] task 0 finished
timeout=4s has expired -> exiting task 0