python多处理starmap进度条

Question

由于我在worker函数中需要几个args，所以我使用starmap，但是如何用tqdm显示进度？

from itertools import repeat
from multiprocessing import Pool

def func(i, a, b, c):
    print(i, a, b, c)

if __name__ == '__main__':  
    pool = Pool(thread_num)
    a, b, c = 'a', 'b', 'c'
    pool.starmap(func, zip(range(100), repeat(a), repeat(b), repeat(c)))
    pool.close()
    pool.join()

那么如何用tqdm来显示这个pregress呢？

Answer 1

您应该创建一个进程来监视其他进程传递的信号并更新您的tqdm。 最简单的例子：

from multiprocessing import Queue, Pool, Process

def listener(q, num):
    tbar = tdqm(total = num)
    for i in iter(q.get, None):
        tbar.update()
    tbar.close()

def worker(q):
    do something.
    queue.put(1)

if __name__ == "__main__":
    ....
    q.put(None)  #when you finish your work, put a None signal to finish the listener.