How can I redirect prints that occur within a multiprocessing Pool into a StringIO()
I am redirecting the sys.stdout
into a StringIO()
, this works well as long as I don't use pool
from the multiprocessing
library.
This toy code is an example:
import io
import sys
from multiprocessing import Pool
print_file = io.StringIO()
sys.stdout = print_file
def a_print_func(some_string):
print(some_string)
pool = Pool(2)
out = pool.map(a_print_func, [['test_1','test_1'],['test_2','test_2']])
a_print_func('no_pool')
print('no_pool, no_func')
fd = open('file.txt', 'w')
fd.write(print_file.getvalue())
fd.close()
file.txt
only contains:
no_pool
no_pool, no_func
instead of:
test_1
test_1
test_2
test_2
no_pool
no_pool, no_func
Here's a solution for directing the output from all the child processes to a single file, using an initializer:
import io
import sys
from multiprocessing import Pool
print_file = io.StringIO()
print_file = open("file.txt", "w")
def a_print_func(some_string):
print(some_string)
def foo(*args):
sys.stdout = print_file
pool = Pool(2, initializer = foo)
out = pool.map(a_print_func, [['test_1','test_1'],['test_2','test_2']])
a_print_func('no_pool')
print('no_pool, no_func')
The output of the program is
no_pool
no_pool, no_func
And, the content of file.txt
at the end of the execution is:
['test_1', 'test_1']
['test_2', 'test_2']
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.