Purpose of using with for python multiprocessing pool

Question

I have been learning the multi process module in python and I notice in the documentation they use with Pool ...

with Pool(processes=4) as pool: pool.map(function,item) vs.

pool=Pool(4) pool.map(function,item)

But in all the examples I have been seeing, and practice I have just been using pool=Pool(#). What is the difference and purpose of the with. I know that with is used to wrap the execution of a block with methods defined by a context manager but what does it achieve for Pool. I think I am just not understanding Pool.

Answer 1

with is used in python for context management.

When used with pool, it's the equivalent of calling pool.close() after the map or apply method. If not called, you'll end up with a whole lot of ghost processes on your machine.

Another example, is the usage of with while handling files, thus implicitly calling f.close()

Answer 2

It just implicitly calls the close method:

New in version 3.3: Connection objects now support the context management protocol – see Context Manager Types. __enter__() returns the connection object, and __exit__() calls close() .

Answer 3

If you read the source code of the multiprocessing.pools.Pool class you'll see that it has:

def __exit__(self, exc_type, exc_val, exc_tb):
    self.terminate()

So it simply calls the terminate() method for you when the context manager ends.

The terminate() method stops the worker processes immediately without completing outstanding work, while the close() method only prevents any more tasks from being submitted to the pool.

From the documentation :

New in version 3.3: Pool objects now support the context management protocol – see Context Manager Types. __enter__() returns the pool object, and __exit__() calls terminate() .