[英]Reducing function call overhead in python
I developed an application which simulates N robots moving in a grid which try to maximize the amount of visited grid cells in a limited amount of steps, meeting in a goal point. 我开发了一个应用程序,它模拟在网格中移动的N个机器人,这些机器人试图在有限的步骤中最大化访问的网格单元的数量,在目标点进行会议。 It all works correctly, but is horrible slow.
这一切都正常,但是很慢。 It is currently python+numpy+mathplotlib.
它目前是python + numpy + mathplotlib。
Maximum robots there can be have a soft limit of 100 (if it can get higher, it is nice to have). 那里的最大机器人可以有100的软限制(如果它可以变得更高,那么它很好)。
To do that, I do the following, simplified: 为此,我做了以下简化:
while steps > 0:
for robot in robots:
agent.calc(robot,steps)
A robot is a 1x2 numpy array (x-and-y-coordinates). 机器人是1x2 numpy数组(x和y坐标)。
The agent here decides what to do. 这里的代理人决定做什么。 Since I need to switch the tactic and strategy on the fly, I cannot move that logic.
由于我需要动态切换策略和策略,我无法改变这种逻辑。
agent.calc updates a robot in place, one after another. agent.calc一个接一个地更新机器人。
cProfiling it returns the following . cProfiling它返回以下内容 。 Extracting the top
提取顶部
39014272 function calls (39010490 primitive calls) in 150.314 seconds
Ordered by: internal time
ncalls tottime percall cumtime percall filename:lineno(function)
12417735 62.807 0.000 62.807 0.000 distance.py:8(taxicab_distance)
124596 36.882 0.000 36.882 0.000 {numpy.core.multiarray.array}
113657 30.204 0.000 100.800 0.001 logical_agent.py:16(choose_max_distance_to...)
12417013 6.579 0.000 69.384 0.000 squaregrid.py:30(distance)
113700 2.900 0.000 109.769 0.001 logical_agent.py:73(calc)
11652363 2.625 0.000 2.625 0.000 {method 'append' of 'list' objects}
161849 1.653 0.000 1.653 0.000 distance.py:11(euclidean_distance)
113664 1.632 0.000 1.632 0.000 {sorted}
114834 1.185 0.000 1.185 0.000 {method 'keys' of 'dict' objects}
113700 0.695 0.000 1.134 0.000 squaregrid.py:19(neighbours)
I implemented different environments for the robots, the most important is squaregird . 我为机器人实施了不同的环境,最重要的是方形 。 Every environment has its own distance function, since I intended to use different metrics, ie Manhattan/taxicab and euclidean.
每个环境都有自己的距离函数,因为我打算使用不同的指标,即曼哈顿/出租车和欧几里德。 I extracted the distance function into an own distance.py file, since I use it in several occasions.
我将距离函数提取到一个自己的distance.py文件中,因为我在几次使用它。
One can see that taxicab_distance is called alot, since the agent needs to evaluate the distances of a robots four neighbours and itself to a goal point to see whether the next position can still reach the goal and to maximize the distance to all other robots as a optimizing heuristics. 可以看出, taxicab_distance被称为很多,因为代理需要评估机器人四个邻居及其自身到目标点的距离,以查看下一个位置是否仍然可以到达目标并最大化到所有其他机器人的距离优化启发式。
The function does not do anything fancy, just 该功能不做任何花哨的事情
def taxicab_distance(u, v):
return np.abs(u[0] - v[0]) + np.abs(u[1] - v[1])
I know that python has a pretty high function call overhead, and I assume that that hits the performance. 我知道python具有相当高的函数调用开销,我认为这会影响性能。 The {numpy.core.multiarray.array} can be ignored, I think I know what I am doing wrong there.
{numpy.core.multiarray.array}可以忽略,我想我知道我在那里做错了什么。
Distance call chain: agent -> environment.distance -> taxicab_distance 距离呼叫链:代理 - > environment.distance - > taxicab_distance
The question is, how can I reduce the overhead of calling the function? 问题是, 如何减少调用函数的开销? I strongly considered using pythons c extensibility, cython, to be more concrete.
我强烈考虑使用pythons c extensibility,cython,更具体。 Will it work?
它会起作用吗? can there be another reason why it is so slow?
还有另一个原因导致它如此缓慢吗?
First, I'd rewrite it into: 首先,我将其重写为:
def taxicab_distance(u, v):
return np.sum(np.abs(u - v))
Can you compute taxicab_distance
for many robots at once? 你能一次为很多机器人计算
taxicab_distance
吗?
I benchmarked it with inlining, and it took off ~15sec. 我用内联对它进行了基准测试,它起飞了〜15秒。 In the end, I rewrote the number crunching in C++ and used cython for integrating.
最后,我重写了C ++中的数字运算并使用cython进行集成。 After that, it took only 1 second.
之后,只花了1秒钟。
EDIT: cpython -> cython 编辑:cpython - > cython
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.