简体   繁体   中英

How to redirect print output from multiprocessing Pool

How can I redirect prints that occur within a multiprocessing Pool into a StringIO()

I am redirecting the sys.stdout into a StringIO() , this works well as long as I don't use pool from the multiprocessing library.

This toy code is an example:

import io
import sys
from multiprocessing import Pool

print_file = io.StringIO()
sys.stdout = print_file

def a_print_func(some_string):
    print(some_string)

pool = Pool(2)  
out = pool.map(a_print_func, [['test_1','test_1'],['test_2','test_2']])

a_print_func('no_pool')
print('no_pool, no_func')

fd = open('file.txt', 'w')
fd.write(print_file.getvalue())
fd.close()

file.txt only contains:

no_pool
no_pool, no_func

instead of:

test_1
test_1
test_2
test_2
no_pool
no_pool, no_func

Here's a solution for directing the output from all the child processes to a single file, using an initializer:

import io
import sys
from multiprocessing import Pool

print_file = io.StringIO()

print_file = open("file.txt", "w")

def a_print_func(some_string):
    print(some_string)

def foo(*args):
    sys.stdout = print_file

pool = Pool(2, initializer = foo)  
out = pool.map(a_print_func, [['test_1','test_1'],['test_2','test_2']])

a_print_func('no_pool')
print('no_pool, no_func')

The output of the program is

no_pool
no_pool, no_func

And, the content of file.txt at the end of the execution is:

['test_1', 'test_1']
['test_2', 'test_2']

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM