[英]How to pass multiprocessing.Pool instance to apply_async callback function?
Here is my prime factorization program,i added a callback function in pool.apply_async(findK, args=(N,begin,end))
,a message prompt out prime factorization is over
when factorization is over,it works fine. 这是我的主要分解程序,我在pool.apply_async(findK, args=(N,begin,end))
添加了一个回调函数,当分解prime factorization is over
时,一个消息提示出prime factorization is over
结束,它工作正常。
import math
import multiprocessing
def findK(N,begin,end):
for k in range(begin,end):
if N% k == 0:
print(N,"=" ,k ,"*", N/k)
return True
return False
def prompt(result):
if result:
print("prime factorization is over")
def mainFun(N,process_num):
pool = multiprocessing.Pool(process_num)
for i in range(process_num):
if i ==0 :
begin =2
else:
begin = int(math.sqrt(N)/process_num*i)+1
end = int(math.sqrt(N)/process_num*(i+1))
pool.apply_async(findK, args=(N,begin,end) , callback = prompt)
pool.close()
pool.join()
if __name__ == "__main__":
N = 684568031001583853
process_num = 16
mainFun(N,process_num)
Now i want to change the callback function in apply_async,to change prompt into a shutdown function to kill all other process. 现在我想更改apply_async中的回调函数,将提示更改为关闭函数以终止所有其他进程。
def prompt(result):
if result:
pool.terminate()
The pool instance is not defined in prompt scope or passed into prompt. 池实例未在提示范围中定义或传递到提示中。
pool.terminate()
can't work in prompt function. pool.terminate()
无法在提示函数中工作。
How to pass multiprocessing.Pool instance to apply_async'callback function ? 如何将multiprocessing.Pool实例传递给apply_async'callback函数?
(I have made it done in class format,just to add a class method and call self.pool.terminate can kill all other process, how to do the job in function format?) (我已经完成了类格式化,只是添加一个类方法并调用self.pool.terminate可以杀死所有其他进程,如何以函数格式完成工作?)
if not set pool as global variable, can pool be passed into callback function? 如果没有将pool设置为全局变量,可以将池传递给回调函数吗?
Passing extra arguments to the callback function is not supported. 不支持将额外参数传递给回调函数。 Yet you have plenty of elegant ways to workaround that. 然而,你有很多优雅的方法来解决这个问题。
You can encapsulate your pool logic into an object: 您可以将池逻辑封装到对象中:
class Executor:
def __init__(self, process_num):
self.pool = multiprocessing.Pool(process_num)
def prompt(self, result):
if result:
print("prime factorization is over")
self.pool.terminate()
def schedule(self, function, args):
self.pool.apply_async(function, args=args, callback=self.prompt)
def wait(self):
self.pool.close()
self.pool.join()
def main(N,process_num):
executor = Executor(process_num)
for i in range(process_num):
...
executor.schedule(findK, (N,begin,end))
executor.wait()
Or you can use the concurrent.futures.Executor implementation which returns a Future
object. 或者您可以使用concurrent.futures.Executor实现,该实现返回Future
对象。 You just append the pool to the Future
object before setting the callback. 您只需在设置回调之前将池附加到Future
对象。
def prompt(future):
if future.result():
print("prime factorization is over")
future.pool_executor.shutdown(wait=False)
def main(N,process_num):
executor = concurrent.futures.ProcessPoolExecutor(max_workers=process_num)
for i in range(process_num):
...
future = executor.submit(findK, N,begin,end)
future.pool_executor = executor
future.add_done_callback(prompt)
You can simply define a local close
function as a callback: 您可以简单地将本地close
函数定义为回调:
import math
import multiprocessing
def findK(N, begin, end):
for k in range(begin, end):
if N % k == 0:
print(N, "=", k, "*", N / k)
return True
return False
def mainFun(N, process_num):
pool = multiprocessing.Pool(process_num)
def close(result):
if result:
print("prime factorization is over")
pool.terminate()
for i in range(process_num):
if i == 0:
begin = 2
else:
begin = int(math.sqrt(N) / process_num * i) + 1
end = int(math.sqrt(N) / process_num * (i + 1))
pool.apply_async(findK, args=(N, begin, end), callback=close)
pool.close()
pool.join()
if __name__ == "__main__":
N = 684568031001583853
process_num = 16
mainFun(N, process_num)
You can also use a partial
function from functool
, with 你也可以使用functool
的partial
功能
import functools
def close_pool(pool, results):
if result:
pool.terminate()
def mainFun(N, process_num):
pool = multiprocessing.Pool(process_num)
close = funtools.partial(close_pool, pool)
....
You need to have pool
end up in prompt
's environment. 你需要让pool
在prompt
环境中结束。 One possibility is to move pool
into the global scope (though this isn't really best-practice). 一种可能性是将pool
转移到全球范围(尽管这不是最佳实践)。 This appears to work: 这似乎有效:
import math
import multiprocessing
pool = None
def findK(N,begin,end):
for k in range(begin,end):
if N% k == 0:
print(N,"=" ,k ,"*", N/k)
return True
return False
def prompt(result):
if result:
print("prime factorization is over")
pool.terminate()
def mainFun(N,process_num):
global pool
pool = multiprocessing.Pool(process_num)
for i in range(process_num):
if i ==0 :
begin =2
else:
begin = int(math.sqrt(N)/process_num*i)+1
end = int(math.sqrt(N)/process_num*(i+1))
pool.apply_async(findK, args=(N,begin,end) , callback = prompt)
pool.close()
pool.join()
if __name__ == "__main__":
N = 684568031001583853
process_num = 16
mainFun(N,process_num)
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.