順序流程的兩個並行流程

Question

我有幾個進程，例如A_step1，A_step2，B_step1，B_step2 ...它們必須以必須在step2開始運行之前必須完成step1的方式運行。 這是我所做的：

from subprocess import check_call
check_call(A_step1)
check_call(A_step2)
check_call(B_step1)
check_call(B_step2)

但是，我希望A和B進程並行運行。 無論如何，要在Python中實現這一目標？

非常感謝

Answer 1

您可能可以將相關進程放入函數中，然后異步運行它們。 對於異步部分，我建議使用多處理模塊

Answer 2

一種常見的策略是使用隊列作為一種機制，以允許協調員（通常是您的主要流程）分配工作，並作為一種方法，允許工人在完成某件事時告訴協調員。

這是一個簡化的示例。 您可以嘗試隨機的睡眠時間，以說服自己，直到兩個工作人員都完成了第一步的工作之后，才能開始第二步的工作。

from multiprocessing import Process, Manager
from time import sleep
from random import randint

def main():

    # Some queues so that we can tell the workers to advance
    # to the next step, and so that the workers to tell
    # us when they have completed a step.
    workQA = Manager().Queue()
    workQB = Manager().Queue()
    progQ = Manager().Queue()

    # Start the worker processes.
    pA = Process(target = workerA, args = (workQA, progQ))
    pB = Process(target = workerB, args = (workQB, progQ))
    pA.start()
    pB.start()

    # Step through some work.
    for step in (1, 2):
        workQA.put(step)
        workQB.put(step)
        done = []
        while True:
            item_done = progQ.get()
            print item_done
            done.append(item_done)
            if len(done) == 2:
                break

    # Tell the workers to stop and wait for everything to wrap up.
    workQA.put('stop')
    workQB.put('stop')
    pA.join()
    pB.join()

def workerA(workQ, progQ):
    do_work('A', workQ, progQ)

def workerB(workQ, progQ):
    do_work('B', workQ, progQ)

def do_work(worker, workQ, progQ):
    # Of course, in your real code the two workers won't
    # be doing the same thing.
    while True:
        step = workQ.get()
        if step == 1:
            do_step(worker, step, progQ)
        elif step == 2:
            do_step(worker, step, progQ)
        else:
            return

def do_step(worker, step, progQ):
    n = randint(1, 5)
    msg = 'worker={} step={} sleep={}'.format(worker, step, n)
    sleep(n)
    progQ.put(msg)   

main()

輸出示例：

worker=B step=1 sleep=2
worker=A step=1 sleep=4
worker=A step=2 sleep=1
worker=B step=2 sleep=3