如何將 Popen 對象傳遞給 concurrent.futures.ProcessPoolExecutor

Question

正如您在下面看到的，我有兩個並行運行的 Popen 對象，但是在每個進程完成后，我想使用從每個進程單獨接收的數據執行一些后處理任務，但我希望后處理任務並行執行但我得到在我調用 executor.map function 的那一刻卡住了，我觀察到 CPU 利用率在一段時間內達到了 100%，后來它下降了，但沒有取得任何結果。 我被無限期地卡住了，這個過程繼續運行。 它甚至不打印Inside letsee 。

subProcess1 = Popen(cmd1, stdout=PIPE, stderr=PIPE, shell=True)
subProcess2 = Popen(cmd2, stdout=PIPE, stderr=PIPE, shell=True)

def letsSee(subProcessId):
    print("Inside letsee")
    while True:
        stdoutVar = subProcessId.stdout.readline()
        if stdoutVar == b'' and subProcessId.poll() is not None:
            break
        if stdoutVar:            
            print(subProcessId, type(stdoutVar), stdoutVar)            
        rc = subProcessId.poll()
    return "Yes"

if __name__ == '__main__':
    with concurrent.futures.ProcessPoolExecutor() as executor:
        print("here")
        xList = [subProcess1, subProcess2]
        print(xList) #output is [<subprocess.Popen object at 0x01365290>, <subprocess.Popen object at 0x01365292>]
        results = executor.map(letsSee, xList)

Answer 1

這對你來說可以接受嗎？

def letsSee(command):
    print("Inside letsee")
    subProcessId = Popen(command, stdout=PIPE, stderr=PIPE, shell=True)
    while True:
        stdoutVar = subProcessId.stdout.readline()
        if stdoutVar == b'' and subProcessId.poll() is not None:
            break
        if stdoutVar:
            print(subProcessId, type(stdoutVar), stdoutVar)
        rc = subProcessId.poll()
    return "Yes"

if __name__ == '__main__':
    with ProcessPoolExecutor() as executor:
        print("here")
        xList = [cmd1, cmd2]
        print(xList)
        results = executor.map(letsSee, xList)