Why i got the exception when using multiprocessing.Pool?

Question

Here is my code:

import multiprocessing

class Worker:
    def run(self, msg):
        print '%s, it works!' % msg

def start_process():
    print 'Starting',multiprocessing.current_process().name

if __name__=='__main__':
    pool = multiprocessing.Pool(processes=2, initializer=start_process,)
    pool.apply_async(Worker().run, args=('congbo',))
    pool.close()
    pool.join()

But I got a exception in console:

Starting PoolWorker-1
Starting PoolWorker-2
Exception in thread Thread-2:
Traceback (most recent call last):
  File "/usr/lib/python2.7/threading.py", line 551, in __bootstrap_inner
    self.run()
  File "/usr/lib/python2.7/threading.py", line 504, in run
    self.__target(*self.__args, **self.__kwargs)
  File "/usr/lib/python2.7/multiprocessing/pool.py", line 319, in _handle_tasks
    put(task)
PicklingError: Can't pickle <type 'instancemethod'>: attribute lookup __builtin__.instancemethod failed

I just passed a class.func to the pool.apply_async(), but I got exception. And can anybody tell me how to pass the class.func to pool.apply_async(), 3x!!!

Answer 1

Functions that are a part of a class is not called functions, they are called methods, and you can't pass a method of an object instance to apply_async(), you need to use a function.

The reason for this is that multiprocessing is all about doing things in multiple processes, and when you pass an instance method, you pass something that is a part of an object. And that object exists only in one of the processes, so the other processes can't access it.

Change your

class Worker:
    def run(self, msg):
        print '%s, it works!' % msg

to

def run(msg):
    print '%s, it works!' % msg

And use that, and it should work.