并行运行两个不同的进程py 3.8

Question

I want to develop a system that reads input from two devices at the same time. Each process works independently at the moment but since I need to sync them, I want them both to write their output on the same file.

import multiprocessing as mp
from multiprocessing import Process
from multiprocessing import Pool
import time

# running the data aquisition from the screen
def Screen(fname):

    for x in range(1, 9):
        fname.write(str(x)+ '\n')
        fname.flush()
        time.sleep(0.5)
        print(x)

# running the data aquisition from the EEG 
def EEG(fname):

    for y in range(10, 19):
        fname.write(str(y)+ '\n')
        fname.flush()
        time.sleep(0.3)
        print(y)


# main program body #

# open the common file that the processes write to
fname = open('C:/Users/Yaron/Documents/Python Scripts/research/demofile.txt', 'w+')

pool = Pool(processes=2)

p1 = pool.map_async(Screen,fname)
p2 = pool.map_async(EEG,fname)

print ('end')

fname.close()

Answer 1

In multiprocessing, depending on the OS you may not be able to pass an open file handle to the process. Here's code that should work on any OS:

import multiprocessing as mp
import time

def Screen(fname,lock):
    with open(fname,'a') as f:
        for y in range(1,11):
            time.sleep(0.5)
            with lock:
                print(y)
                print(y,file=f,flush=True)

def EEG(fname,lock):
    with open(fname,'a') as f:
        for y in range(11, 21):
            time.sleep(0.3)
            with lock:
                print(y)
                print(y,file=f,flush=True)

if __name__ == '__main__':
    fname = 'demofile.txt'
    lock = mp.Lock()
    with open(fname,'w'): pass  # truncates existing file and closes it
    processes = [mp.Process(target=Screen,args=(fname,lock)),
                 mp.Process(target=EEG,args=(fname,lock))]
    s = time.perf_counter()
    for p in processes:
        p.start()
    for p in processes:
        p.join()
    print (f'end (time={time.perf_counter() - s}s)')

Some notes:

• Open the file in each process. Windows, for example, doesn't fork() the process and doesn't inherit the handle. The handle isn't picklable to pass between processes.
• Open the file for append. Two processes would have two different file pointers. Append makes sure it seeks to the end each time.
• Protect the file accesses with a lock for serialization. Create the lock in the main thread and pass the same Lock to each process.
• Use if __name__ == '__main__': to run one-time code in the main thread. Some OSes import the script in other processes and this protects the code from running multiple times.
• map_async isn't used correctly. It takes an iterable of arguments to pass to the function. Instead, make the two processes, start them, and join them to wait for completion.