append to instance list with multiprocess

Question

I have a class that has a list for one of its attributes. After instantiating the class, I would like to do some calculations then append the results (which are lists themselves) to the list. I'm trying to use multiprocess to speed up the calculations/appending since since the order doesn't matter. The following is a simplified example:

import multiprocessing as mp

class Test(object):
    def __init__(self):
        self.basis = []

    def action(self, i):
        self.basis.append([i])

    def test(self):
        for i in xrange(5):
            self.action(i)
        print(self.basis)

    def test_mp(self):
        jobs = []
        for i in xrange(5):
            job = mp.Process(target=self.action, args=(i,))
            jobs.append(job)
            job.start()
        for j in jobs:
            j.join()
        print(self.basis)

Test().test()
# Returns correct list
Test().test_mp()
# Returns empty (incorrect) list

My approach only seems to work without trying to run in parallel (works for test , but not for test_mp ). I am pretty sure that test_mp isn't working because each process isn't sharing it's own version of self.basis , but I'm not sure how to overcome this. I have looked into making self.basis an mp.Array , but that fails since I am trying to append lists, not just numbers. I have found several other similar questions, but their answers seem to break down when trying to use their solutions within a class like this (pickling errors, etc.)

Answer 1

You could use Manager

import multiprocessing as mp

class Test(object):
    def __init__(self):
        self.basis = mp.Manager().list()

Output:

[[0], [3], [1], [4], [2]]