使用python多重处理同时写入文件

Question

我正在开发一个项目，其中使用约6个传感器将其送入Beaglebone Black，从而将该数据连续保存到6个不同的文件中。 通过另一个SO问题（ https://stackoverflow.com/a/36634587/2615940 ），我了解到多处理模块将为我执行此操作，但是在运行新代码时，我仅获得1个文件而不是6个文件。如何修改此代码以获得所需的6个结果文件？

*我已经根据下面的skrrgwasme的建议编辑了文件，使其包含Manager ，但是现在代码运行了，什么也没有产生。 没有错误，没有文件。 刚运行。

编码：

import Queue
import multiprocessing
import time

def emgacq(kill_queue, f_name, adcpin):
     with open(f_name, '+') as f:
        while True:
            try:
                val = kill_queue.get(block = False)
                if val == STOP:
                    return
            except Queue.Empty:
                pass
            an_val = ADC.read(adcpin) * 1.8
            f.write("{}\t{}\n".format(ms, an_val))
def main():    
    #Timing stuff
    start = time.time()
    elapsed_seconds = time.time() - start
    ms = elapsed_seconds * 1000

    #Multiprcessing settings
    pool = multiprocessing.Pool()
    m = multiprocessing.Manager()
    kill_queue = m.Queue()            

    #All the arguments we need run thru emgacq()
    arg_list = [
        (kill_queue, 'HamLeft', 'AIN1'),
        (kill_queue, 'HamRight', 'AIN2'),
        (kill_queue, 'QuadLeft', 'AIN3'),
        (kill_queue, 'QuadRight', 'AIN4'),
        (kill_queue, 'GastLeft', 'AIN5'),
        (kill_queue, 'GastRight',  'AIN6'),
        ]

    for a in arg_list:
        pool.apply_async(emgacq, args=a) 

    try: 
        while True:
            time.sleep(60) 
    except KeyboardInterrupt:
        for a in arg_list:
            kill_queue.put(STOP)
        pool.close()
        pool.join()
        raise f.close()

if __name__ == "__main__":
    main()

Answer 1

您的主要问题是子流程函数的参数列表不正确：

f_list = [
    emgacq(kill_queue, 'HamLeft', 'AIN1'), 
    # this calls the emgacq function right here - blocking the rest of your 
    # script's execution

另外，您的apply_async调用是错误的：

for f in f_list:
    pool.apply_async(f, args=(kill_queue)) 
    # f is not a function here - the arguments to the apply_async function
    # should be the one function you want to call followed by a tuple of 
    # arguments that should be provided to it

您需要这样做，它还包括队列manager （请参阅https://stackoverflow.com/a/9928191/2615940 ），并将所有代码放入main函数中：

# put your imports here
# followed by the definition of the emgacq function

def main():

    #Timing stuff
    start = time.time()
    elapsed_seconds = time.time() - start
    ms = elapsed_seconds * 1000

    pool = multiprocessing.Pool()
    m = multiprocessing.Manager()
    kill_queue = m.Queue()

    arg_list = [
            (kill_queue, 'HamLeft', 'AIN1'), 
            (kill_queue, 'HamRight', 'AIN2'),
            (kill_queue, 'QuadLeft', 'AIN3'),
            (kill_queue, 'QuadRight', 'AIN4'),
            (kill_queue, 'GastLeft', 'AIN5'),
            (kill_queue, 'GastRight',  'AIN6'),
        ]

    for a in arg_list:
        pool.apply_async(emgacq, args=a)
        # this will call the emgacq function with the arguments provided in "a"

if __name__ == "__main__":
    # you want to have all of your code in a function, because the workers
    # will start by importing the main module they are executing from,
    # and you don't want them to execute that code all over again
    main()