多进程pool.apply_async执行序列

Question

我正在学习在网站某处找到的多处理和运行代码（我添加了print("check...") ），以观察使用apply_async运行的代码序列。 下面是代码：

import multiprocessing
import time
def func(msg):
  for i in range(3):
    print(msg)
    time.sleep(1)
if __name__ == "__main__":
  pool = multiprocessing.Pool(processes=8)
  for i in range(10):
    msg = "hello %d" %(i)
    pool.apply_async(func, (msg, ))
    print('check whether block' + str(i))
  print('check')
  pool.close()
  pool.join()
  print("Sub-process(es) done.")

运行它时，得到以下输出：

hello 0
hello 2
hello 4
hello 1
hello 3
hello 5
hello 6
hello 7
check whether block0
check whether block1
check whether block2
check whether block3
check whether block4
check whether block5
check whether block6
check whether block7
check whether block8
check whether block9
check
hello 0
hello 4
hello 1
hello 5
hello 2
hello 3
hello 6
hello 7
hello 4
hello 0
hello 6
hello 3
hello 5
hello 2
hello 7
hello 1
hello 8
hello 9
hello 9
hello 8
hello 9
hello 8
Sub-process(es) done.

但是我希望看到以下内容：

hello 0
check whether block0
hello 2
check whether block2
hello 4
check whether block4
hello 1
check whether block1
hello 3
check whether block3
hello 5
check whether block5
hello 6
check whether block6
hello 7
check whether block7
hello 0
hello 4
hello 1
hello 5
hello 2
hello 3
hello 6
hello 7
hello 4
hello 0
hello 6
hello 3
hello 5
hello 2
hello 7
hello 1
hello 8
check whether block8
hello 9
check whether block9
hello 9
hello 8
hello 9
hello 8
check
Sub-process(es) done.

基本上我认为应该在每个pool.apply_async执行前8个print('check whether...') 。 释放2个步骤后，将打印8和9。 并在最后打印支票。 我所看到的似乎证明我错了，但我不知道代码序列的执行方式。

Answer 1

首先，不要使用time.sleep进行类似的测试，因为python很聪明并且可能会做一些偷偷摸摸的事情。 做类似sum(range(10_000_000)) ，以确保python（和您的操作系统）没有将cpu用于其他操作。

其次，每次运行.apply_async() ，都会在不阻塞程序的情况下将任务添加到队列中，因此查看所有check ...是完全正常的check ...因为您实际上要检查的是是否添加了它们。排队。

第三，启动过程需要时间，但是将任务添加到队列是瞬时的，这就是为什么您在其余所有任务之前都看到所有check ...原因。

我编辑了您的程序，所以它更加明显。

def func(msg):
    for i in range(3):
        print(msg)
        sum(range(10_000_000))


if __name__ == "__main__":
    pool = multiprocessing.Pool(processes=8)
    for i in range(10):
        msg = f"hello {i: >{i+1}}"
        pool.apply_async(func, (msg, ))
        print('check whether block' + str(i))
    print('check')
    pool.close()
    pool.join()
    print("Sub-process(es) done.")

check whether block0             # <- adding task 0 to the queue
check whether block1             # <- adding task 1 to the queue
check whether block2             # <- adding task 2 to the queue
check whether block3             # <- adding task 3 to the queue
check whether block4             # <- adding task 4 to the queue
check whether block5             # <- adding task 5 to the queue
check whether block6             # <- adding task 6 to the queue
check whether block7             # <- adding task 7 to the queue
check whether block8             # <- adding task 8 to the queue
check whether block9             # <- adding task 9 to the queue
check                            #  ALL TASKS ADDED TO THE QUEUE
hello 0                          # -> Process 0 has been created to do task 0
hello  1                         # -> Process 1 has been created to do task 1
hello   2                        # -> Process 2 has been created to do task 2
hello    3                       # -> Process 3 has been created to do task 3
hello     4                      # -> Process 4 has been created to do task 4
hello      5                     # -> Process 5 has been created to do task 5
hello       6                    # -> Process 6 has been created to do task 6
hello        7                   # -> Process 7 has been created to do task 7
hello 0                          # -> Task is partly done
hello        7                   # -> Task is partly done
hello      5                     # -> Task is partly done
hello     4                      # -> Task is partly done
hello    3                       # -> Task is partly done
hello       6                    # -> Task is partly done
hello   2                        # -> Task is partly done
hello  1                         # -> Task is partly done
hello        7                   # -> Task is done
hello 0                          # -> Task is done
hello      5                     # -> Task is done
hello       6                    # -> Task is done
hello   2                        # -> Task is done
hello     4                      # -> Task is done
hello  1                         # -> Task is done
hello    3                       # -> Task is done
hello         8                  # -> Process 0 has been given task 8
hello          9                 # -> Process 7 has been given task 9
hello         8                  # -> Task is partly done
hello          9                 # -> Task is partly done
hello         8                  # -> Task is done
hello          9                 # -> Task is done
Sub-process(es) done.            # EVERYTHING DONE

Check whether block{X}第一个hello {X+1} Check whether block{X}之后Check whether block{X}可能是由于您控制台输出不同进程和刷新的速度。