Unexpected results of measures with multiprocessing in Python

Question

I am trying to speed up a little calculations of Rosenbrock function with concurrent genetic algorithm vs linear genetic algorithm. I started play around with threading and multiprocessing Python libraries and I found a way how to do it, BUT (there is always a 'but') I discovered completely unexpected behaviour in evaluation.

I measured calculations for 2D Rosenbrock (or any bigger dimension) for population in range[5 - 500000], 10 tests per population. What is wrong?

1 process is much faster than iter algorithm, even 50% less time spend on calculations which seems completely wrong.

Do you have any idea why I have much gain between it? One process should make calculations in similar time as iter algorithm (even worse because of resources needed to run process, right?)

You can see full results on link ('n' means dimension of Rosenbrock)

#!/usr/bin/python
import scipy
import multiprocessing
from timeit import default_timer as timer
import math

def rosenbrock(x_1, x_2):
    return 100*(x_2-x_1**2)**2 + (1-x_1)**2

def n_rosenbrock(X):
    sum_r = 0
    for i in range(len(X)-1):
        sum_r += rosenbrock(X[i], X[i+1])
    return sum_r

def evaluation(shared_population, shared_fitnesses, nr_of_genes, x_1, x_2):
    for i in range(x_1, x_2, nr_of_genes):
        result = n_rosenbrock(shared_fitnesses[i:i+nr_of_genes])
        shared_fitnesses[int(i/nr_of_genes)] = result

if __name__ == '__main__':
    min_x = -5
    max_x = 5
    cores = 1

    POP_SIZES = [5, 20, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 25000, 50000, 100000, 150000, 200000, 250000, 300000, 350000, 400000, 450000, 500000]
    iters_time = []
    proc_eval_time = []


    for idp, pop_size in enumerate(POP_SIZES):
        for nr_of_genes in range(2, 3):
            population = scipy.random.uniform(min_x, max_x, (pop_size * nr_of_genes))
            shared_population = multiprocessing.Array('f', scipy.reshape(population, pop_size*nr_of_genes), lock=False)
            shared_fitnesses = multiprocessing.Array('f', pop_size, lock=False)

            indexes = [int(pop_size/cores)] * cores
            for x in range(int(pop_size%cores)):
                indexes[x] += 1
            test_c = 10
            process_eval_time = 0
            process_sel_time = 0

            iter_time = 0

            print("Iter", idp)
            iter_population = scipy.reshape(population, pop_size*nr_of_genes)
            iter_fitnesses = scipy.zeros(pop_size)
            for _ in range(test_c):
                iter_timer_start = timer()
                for i in range(0,len(iter_population),nr_of_genes):
                    result = n_rosenbrock(iter_population[i:i+nr_of_genes])
                    iter_fitnesses[int(i/nr_of_genes)] = result
                iter_timer_stop = timer()
                iter_time += (iter_timer_stop-iter_timer_start)
            iters_time.append(iter_time/test_c)

            print("Process", idp)
            for _ in range(test_c):
                processes = scipy.empty(cores, dtype=multiprocessing.Process)
                for idx in range(cores):
                    x_1 = sum(indexes[:idx]) * nr_of_genes
                    x_2 = x_1 + indexes[idx] * nr_of_genes
                    args = (shared_population, shared_fitnesses, nr_of_genes, x_1, x_2)
                    process = multiprocessing.Process(target=evaluation, args=args)
                    processes[idx] = process
                process_eval_start = timer()
                for p in processes:
                    p.start()
                for p in processes:
                    p.join()
                process_eval_stop = timer()
                process_eval_time += (process_eval_stop-process_eval_start)
            proc_eval_time.append(process_eval_time/test_c)

    print("iters_time", iters_time)
    print("process_eval_time", proc_eval_time)

Answer 1

It seems like your comparisons might not be valid. I'd suggest organizing the code like this:

def do_iter(x, y, z):
    ...

def do_multiproc(x, y, z):
    ...

for x in population_sizes:
    timeit.timeit('do_iter(x, y, z)')
    timeit.timeit('do_multiproc(x, y, z)')

This code obviously won't run. The point is that all the setup and processing involved with each method should be fully encapsulated in the do_x function for that method. The do_x functions should take the same args and otherwise be as similar as possible.

Also, it looks like you're testing each combination of args 10 times, which might not be enough to get accurate timings. timeit.timeit() defaults to 1,000,000 iterations.