[英]Is `built-in method numpy.core._multiarray_umath.implement_array_function` a performance bottleneck?
I'm using numpy v1.18.2
in some simulations, and using inbuilt functions such as np.unique
, np.diff
and np.interp
.我在一些模拟中使用
numpy v1.18.2
,并使用np.unique
、 np.diff
和np.interp
等内置函数。 I'm using these functions on standard objects, ie lists or numpy arrays.我在标准对象上使用这些函数,即列表或 numpy arrays。
When I checked with cProfile
, I saw that these functions make a call to an built-in method numpy.core._multiarray_umath.implement_array_function
and that this method accounts for 32.5%
of my runtime.当我检查
cProfile
时,我看到这些函数调用了一个内置方法numpy.core._multiarray_umath.implement_array_function
并且这个方法占了我运行时间的32.5%
。 To my understanding this is a wrapper that performs some checks to make sure the that the arguments passed to the function are compatible with the function.据我了解,这是一个包装器,它执行一些检查以确保 arguments 传递给 function 与该功能兼容。
I have two questions:我有两个问题:
implement_array_function
) actually taking up so much time or is it actually the operations I'm doing ( np.unique
, np.diff
, np.interp
) that is actually taking up all this time?implement_array_function
)实际上占用了这么多时间,还是实际上我正在做的操作( np.unique
, np.diff
, np.interp
)实际上占用了所有这些时间? That is, am I misinterpreting the cProfile output? I already saw this question ( what is numpy.core._multiarray_umath.implement_array_function and why it costs lots of time? ), but I was not able to understand what exactly the function is or does.我已经看到了这个问题(什么是 numpy.core._multiarray_umath.implement_array_function 以及为什么要花费大量时间? ),但我无法理解 function 到底是什么或做什么。 I also tried to understand NEP 18 , but couldn't make out how to exactly solve the issue.
我也试图理解NEP 18 ,但无法弄清楚如何准确解决这个问题。 Please fill in any gaps in my knowledge and correct any misunderstandings.
请补充我的知识空白并纠正任何误解。 Also I'd appreciate if someone can explain this to me like I'm 5 (r/explainlikeimfive/) instead of assuming developer level knowledge of python.
另外,如果有人能像我 5 岁(r/explainlikeimfive/)一样向我解释这一点,而不是假设 python 的开发人员级别知识,我将不胜感激。
All the information below is taken from NEP 18 .以下所有信息均取自NEP 18 。
Is this function (
implement_array_function
) actually taking up so much time or is it actually the operations I'm doing (np.unique
,np.diff
,np.interp
) that is actually taking up all this time?这是 function (
implement_array_function
)实际上占用了这么多时间,还是实际上我正在做的操作(np.unique
,np.diff
,np.interp
)实际上占用了所有这些时间?
As @hpaulj correctly mentioned in the comment, the overhead of the dispatcher adds 2-3 microseconds to each numpy function call.正如@hpaulj 在评论中正确提到的那样,调度程序的开销为每个 numpy function 调用增加了 2-3 微秒。 This will probably be shorten to 0.5-1 microseconds once it is implemented in C.
一旦在 C 中实现,这可能会缩短到 0.5-1 微秒。 See here .
见这里。
Is there any way to disable it/bypass it
有什么方法可以禁用它/绕过它
Yes, from NumPy 1.17, you can set the environment variable NUMPY_EXPERIMENTAL_ARRAY_FUNCTION
to 0 ( before importing numpy ) and this will disable the use of implement_array_function
(See here ).是的,从 NumPy 1.17 开始,您可以将环境变量
NUMPY_EXPERIMENTAL_ARRAY_FUNCTION
设置为 0(在导入 numpy 之前),这将禁用implement_array_function
的使用(参见此处)。 Something like就像是
import os
os.environ['NUMPY_EXPERIMENTAL_ARRAY_FUNCTION'] = '0'
import numpy as np
However, disabling it probably would not give you any notable performance improvement as the overhead of it is just few microseconds, and this will be the default in later numpy version too.但是,禁用它可能不会给您带来任何显着的性能改进,因为它的开销只有几微秒,这也将是后续 numpy 版本的默认设置。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.