创建可重复周期Python的工作程序

Question

How I could use workers (multi processing) when for example I could use 10 at one time at most but I need them to do ~ 150 tasks.

def main():
    for i in serverinfo :
        grabfile(i,serverinfo[i]["ip"],serverinfo[i]["psw"])

grabfile is task which I want to do with those ~10 workers, but I really don't understand how to make them work together that for example they all starter from task numbers 1-10 and then when they done they would get new tasks to do tasks 11-20 PS I refer task number as "i" in code snipet

Answer 1

Use the ThreadPoolExecuter from cuncurrnt.futures

It should look something like this:

import concurrent.futures

pool = concurrent.futures.ThreadPoolExecutor(10) # max 10 worker threads

for x in data_to_work_on:
    pool.submit(worker_function, x.param1, x.param2)