简体   繁体   中英

python multiprocessing is not using multiple cores

Reading the documentation: https://docs.python.org/2/library/multiprocessing.html I decided to write a cpu intensive code and compare multiprocessing with serial computation. First of all, if this library is using multiprocessing, then why I only see 1 python.exe process? Secondly, why serial computation takes 12 seconds while multiprocessed one takes 22 seconds?

serial code:

from datetime import datetime

def calc_fib(ind):
    fb = 1
    if ind >= 3:
        prev = 1
        i = 2
        while i < ind:
            prev_tmp = fb
            fb += prev
            prev = prev_tmp
            i += 1
    return fb


def long_calc_fib(ind):
    val = 0
    for j in range(500):
        val = calc_fib(ind)
    return val

if __name__ == "__main__":
    t1 = datetime.now()
    for i in range(10):
        tmp = long_calc_fib(10000)
    t2 = datetime.now()
    print str(t2 - t1)

multiprocessing pool code:

from datetime import datetime
from multiprocessing.pool import ThreadPool

def calc_fib(ind):
    fb = 1
    if ind >= 3:
        prev = 1
        i = 2
        while i < ind:
            prev_tmp = fb
            fb += prev
            prev = prev_tmp
            i += 1
    return fb


def long_calc_fib(ind):
    val = 0
    for j in range(500):
        val = calc_fib(ind)
    return val


if __name__ == "__main__":
    t1 = datetime.now()

    pool = ThreadPool(processes=10)
    async_results = []
    for i in range(10):
        async_results.append(pool.apply_async(long_calc_fib, (10000,)))
    for res in async_results:
        tmp = res.get()

    t2 = datetime.now()
    print str(t2 - t1)

My mistake. I must have used Pool instead of ThreadPool. By chaning ThreadPool to Pool, I reduced the time to 3 seconds.

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM