简体   繁体   English

如何将仅关键字 arguments 应用于多处理池中的 function?

[英]How to apply keyword-only arguments to a function in a multiprocessing pool?

I have a function that takes a keyword-only argument and want to run it in a process pool.我有一个 function 需要一个仅关键字参数并希望在进程池中运行它。 How do I pass my entries from an iterable to the function in the process as a keyword argument?如何在过程中将我的条目从可迭代传递到 function 作为关键字参数?

import multiprocessing

greetees = ('Foo', 'Bar')

def greet(*, greetee):
    return f'Hello, {greetee}!'

I tried using multiprocessing.map:我尝试使用 multiprocessing.map:

greetings = multiprocessing.Pool(2).map(greet, greetees)
for greeting in greetings:
    print(greeting)

But that raises an exception, as expected:但这引发了一个例外,正如预期的那样:

multiprocessing.pool.RemoteTraceback: 
"""
Traceback (most recent call last):
  File "/usr/lib/python3.6/multiprocessing/pool.py", line 119, in worker
    result = (True, func(*args, **kwds))
  File "/usr/lib/python3.6/multiprocessing/pool.py", line 44, in mapstar
    return list(map(*args))
TypeError: greet() takes 0 positional arguments but 1 was given
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/bengt/Projekte/gitlab.com/PFASDR/PFASDR.Code.Main/pfasdr/neural/multi_pool_kwargs.py", line 10, in <module>
    greetings = multiprocessing.Pool(2).map(greet, greetees)
  File "/usr/lib/python3.6/multiprocessing/pool.py", line 266, in map
    return self._map_async(func, iterable, mapstar, chunksize).get()
  File "/usr/lib/python3.6/multiprocessing/pool.py", line 644, in get
    raise self._value
TypeError: greet() takes 0 positional arguments but 1 was given

It works fine if I remove the asterisk to not require the arguments to be keyword-only:如果我删除星号以不要求 arguments 仅是关键字,则效果很好:

[...]
def greet(greetee):
    return f'Hello, {greetee}!'
[...]

Output: Output:

Hello, Foo!
Hello, Bar!

A solution here is to use Pool.apply or Pool.apply_async :这里的一个解决方案是使用Pool.applyPool.apply_async

greetings = list(
    multiprocessing.Pool(2).apply(greet, kwds={'greetee': greetees[i]})
    for i in range(len(greetees))
)
for greeting in greetings:
    print(greeting)

Output: Output:

Hello, Foo!
Hello, Bar!

Courtesy of Mad Physicist and this QnA , one can use functools.partial to inject keyword-only arguments into a function:感谢Mad Physicist此 QnA ,可以使用functools.partial将仅关键字 arguments 注入 function:

from functools import partial
greetings = []
for i in range(len(greetees)):
    kwargs = {'greetee': greetees[i]}
    greet_partial = partial(greet, **kwargs)
    greetings.append(multiprocessing.Pool(2).apply(greet_partial))

Or with less variable bleed:或具有较少可变出血:

from functools import partial
greetings = [
    multiprocessing.Pool(2).apply(
        partial(greet, **{'greetee': greetees[i]})
    )
    for i in range(len(greetees))
]

声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.

相关问题 解释 Python 中的仅关键字参数 (VarArgs) - Explain Keyword-Only Arguments (VarArgs) in Python 在修饰函数中强制执行仅关键字 arguments - Enforcing keyword-only arguments in decorated functions 如何使用数据类制作“仅关键字”字段? - How to make “keyword-only” fields with dataclasses? 如何获得仅关键字参数的长度? - how to get length of keyword-only argument? 使用multiprocessing.Pool.map()函数和关键字参数吗? - Using the multiprocessing.Pool.map() function with keyword arguments? 多处理池&#39;apply_async&#39;似乎只调用一次函数 - Multiprocessing pool 'apply_async' only seems to call function once 具有两个参数的函数上的多处理(池) - Multiprocessing (Pool) on a function with two arguments 如何为未来警告更新 pandas DataFrame.drop() - DataFrame.drop 的所有参数(参数“标签”除外)将仅是关键字 - How to update pandas DataFrame.drop() for Future Warning - all arguments of DataFrame.drop except for the argument 'labels' will be keyword-only 如何为python 3.0的仅关键字参数导入__future__? - how to import __future__ for keyword-only argument of python 3.0? 多处理:如何在列表上使用pool.map并使用参数函数? - Multiprocessing: How to use pool.map on a list and function with arguments?
 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM