[英]How to apply keyword-only arguments to a function in a multiprocessing pool?
I have a function that takes a keyword-only argument and want to run it in a process pool.我有一个 function 需要一个仅关键字参数并希望在进程池中运行它。 How do I pass my entries from an iterable to the function in the process as a keyword argument?如何在过程中将我的条目从可迭代传递到 function 作为关键字参数?
import multiprocessing
greetees = ('Foo', 'Bar')
def greet(*, greetee):
return f'Hello, {greetee}!'
I tried using multiprocessing.map:我尝试使用 multiprocessing.map:
greetings = multiprocessing.Pool(2).map(greet, greetees)
for greeting in greetings:
print(greeting)
But that raises an exception, as expected:但这引发了一个例外,正如预期的那样:
multiprocessing.pool.RemoteTraceback:
"""
Traceback (most recent call last):
File "/usr/lib/python3.6/multiprocessing/pool.py", line 119, in worker
result = (True, func(*args, **kwds))
File "/usr/lib/python3.6/multiprocessing/pool.py", line 44, in mapstar
return list(map(*args))
TypeError: greet() takes 0 positional arguments but 1 was given
"""
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/bengt/Projekte/gitlab.com/PFASDR/PFASDR.Code.Main/pfasdr/neural/multi_pool_kwargs.py", line 10, in <module>
greetings = multiprocessing.Pool(2).map(greet, greetees)
File "/usr/lib/python3.6/multiprocessing/pool.py", line 266, in map
return self._map_async(func, iterable, mapstar, chunksize).get()
File "/usr/lib/python3.6/multiprocessing/pool.py", line 644, in get
raise self._value
TypeError: greet() takes 0 positional arguments but 1 was given
It works fine if I remove the asterisk to not require the arguments to be keyword-only:如果我删除星号以不要求 arguments 仅是关键字,则效果很好:
[...]
def greet(greetee):
return f'Hello, {greetee}!'
[...]
Output: Output:
Hello, Foo!
Hello, Bar!
A solution here is to use Pool.apply
or Pool.apply_async
:这里的一个解决方案是使用Pool.apply
或Pool.apply_async
:
greetings = list(
multiprocessing.Pool(2).apply(greet, kwds={'greetee': greetees[i]})
for i in range(len(greetees))
)
for greeting in greetings:
print(greeting)
Output: Output:
Hello, Foo!
Hello, Bar!
Courtesy of Mad Physicist and this QnA , one can use functools.partial to inject keyword-only arguments into a function:感谢Mad Physicist和此 QnA ,可以使用functools.partial将仅关键字 arguments 注入 function:
from functools import partial
greetings = []
for i in range(len(greetees)):
kwargs = {'greetee': greetees[i]}
greet_partial = partial(greet, **kwargs)
greetings.append(multiprocessing.Pool(2).apply(greet_partial))
Or with less variable bleed:或具有较少可变出血:
from functools import partial
greetings = [
multiprocessing.Pool(2).apply(
partial(greet, **{'greetee': greetees[i]})
)
for i in range(len(greetees))
]
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.