简体   繁体   中英

multiprocessing in python runs all the jobs at the same time

I have a function that I want to call 100 times with different input values. I have 8 processors and I want 8 jobs to run simultaneously and after one is finished the other one to start. However, in my script below, python starts all jobs at the same time, what needs to be changed?

global bits
bits = np.zeros(100)

def _multi(bits,idd,some_function):
    bits[idd] = some_function(idd)

jobs = []
for i in range(100):
    d = multiprocessing.Process(target=_multi,args=(bits,i,some_function))
    jobs.append(d)
    d.start()
for job in jobs:
    job.join()

use pool to set the processes.

import multiprocessing as mp
import operator
import os

def process(func,arg):
    pool = mp.Pool(processes=8)

    results=[pool.apply_async(func, args=(arg,i)) for i in range(100)]
    r=[]
    for p in results:
        try:
            r.append(p.get())
        except Exception:
            print("error getting process: %s" % os.getpid())
    return r

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM