ThreadPoolExecutor 生成的期货不会异步运行

Question

我想创建一个在 ThreadPoolExecutor 上运行的期货列表，然后在它们完成评估后立即显示它们中的每一个。

预期结果是：每 3 秒打印一次 0、2、6、12。

但是，我在 12 秒后才得到结果，并且数字是模拟显示的。

from concurrent.futures import ThreadPoolExecutor
import time

def fnc(x, y):
    time.sleep(3)
    return x*y

futures = []
with ThreadPoolExecutor(max_workers=1) as executor:
    for i in range(0, 4):
        print(f"Submitting {i}")
        futures += [executor.submit(fnc, i, i+1)]

for f in futures:
    print(f.result())

Answer 1

构建您提交的期货列表，然后使用as_completed()了解线程何时完成及其结果可用。

from concurrent.futures import ThreadPoolExecutor, as_completed
import time

def fnc(x, y):
    time.sleep(3)
    return x*y

futures = []

with ThreadPoolExecutor() as executor:
    for i in range(0, 4):
        print(f"Submitting {i}")
        futures += [executor.submit(fnc, i, i+1)]
    for future in as_completed(futures):
        print(future.result())

Answer 2

您在ThreadPoolExecutor上下文管理器之外的每个未来调用result方法，当您退出时，它会调用__exit__方法：

def __exit__(self, exc_type, exc_val, exc_tb):
    self.shutdown(wait=True)
    return False

shutdown方法签名是：

shutdown(self, wait=True, *, cancel_futures=False)

文档说：

Args:
     wait: If True then shutdown will not return until all running
           futures have finished executing and the resources used by the
           executor have been reclaimed.
     cancel_futures: If True then shutdown will cancel all pending
           futures. Futures that are completed or running will not be
           cancelled.

我们可以看到默认情况下它会等到所有正在运行的 futures 和它们的资源也停止运行，并且cancel_futures默认值为False ，因此我们不会取消挂起的 futures。

我们可以通过更改fnc来打印值而不是返回值来证明这一点，并且在ThreadPoolExecutor上下文管理器块之后什么也不做：

def fnc(x, y):
    time.sleep(3)
    print(x * y)


futures = []
with ThreadPoolExecutor(max_workers=1) as executor:
    for i in range(0, 4):
        print(f"Submitting {i}")
        futures += [executor.submit(fnc, i, i + 1)]

print("Blah!")

仍然打印值0, 2, 6, 12 。 即使我们只将 function 提交给执行者列表...

---

通过在上下文管理器中移动for循环块来修复它：

from concurrent.futures import ThreadPoolExecutor
import time

def fnc(x, y):
    time.sleep(3)
    return x*y

futures = []
with ThreadPoolExecutor(max_workers=1) as executor:
    for i in range(0, 4):
        print(f"Submitting {i}")
        futures += [executor.submit(fnc, i, i+1)]
    for f in futures:
        print(f.result())

注意，设置max_workers=1本质上是强制程序连续运行，而不是并发运行，在这个程序中，设置max_workers=X会一次打印fnc的X个返回结果。

如果您想在每个结果之间等待三秒钟，则可以将max_workers设置为 1 或将其完全删除。 如果你想每三秒一次打印两个结果 - 设置max_workers=2等等。