[英]Numba and Cython aren't improving the performance compared to CPython significantly, maybe I am using it incorrectly?
BIG EDIT: 大编辑:
================ ================
For the sake of clarity, I am removing the old results and replace it by the more recent results. 为了清楚起见,我删除了旧的结果,并将其替换为最近的结果。 The question is still the same: Am I using both Cython and Numba correctly, and what improvements to the code can be made? 问题仍然是:我是否正确使用Cython和Numba,以及可以对代码进行哪些改进? (I have a newer and more bare-bones temporary IPython notebook with all the code and results here ) (我有一个更新,更裸机临时IPython的笔记本与所有的代码和结果在这里 )
I think I figured out why there was initially no difference between Cython, Numba, and CPython: It was because I fed them 我想我弄清楚为什么Cython,Numba和CPython之间最初没有区别:这是因为我喂它们
numpy arrays as input: numpy数组作为输入:
x = np.asarray([x_i*np.random.randint(8,12)/10 for x_i in range(n)])
instead of lists: 而不是列表:
x = [x_i*random.randint(8,12)/10 for x_i in range(n)]
I replaced the zip()
function by explicit loops, however, it didn't make much of a difference. 我通过显式循环替换了zip()
函数,但是,它并没有太大的区别。 The code would be: 代码是:
def py_lstsqr(x, y):
""" Computes the least-squares solution to a linear matrix equation. """
len_x = len(x)
x_avg = sum(x)/len_x
y_avg = sum(y)/len(y)
var_x = 0
cov_xy = 0
for i in range(len_x):
temp = (x[i] - x_avg)
var_x += temp**2
cov_xy += temp*(y[i] - y_avg)
slope = cov_xy / var_x
y_interc = y_avg - slope*x_avg
return (slope, y_interc)
%load_ext cythonmagic
%%cython
def cy_lstsqr(x, y):
""" Computes the least-squares solution to a linear matrix equation. """
cdef double x_avg, y_avg, var_x, cov_xy,\
slope, y_interc, x_i, y_i
cdef int len_x
len_x = len(x)
x_avg = sum(x)/len_x
y_avg = sum(y)/len(y)
var_x = 0
cov_xy = 0
for i in range(len_x):
temp = (x[i] - x_avg)
var_x += temp**2
cov_xy += temp*(y[i] - y_avg)
slope = cov_xy / var_x
y_interc = y_avg - slope*x_avg
return (slope, y_interc)
from numba import jit
@jit
def numba_lstsqr(x, y):
""" Computes the least-squares solution to a linear matrix equation. """
len_x = len(x)
x_avg = sum(x)/len_x
y_avg = sum(y)/len(y)
var_x = 0
cov_xy = 0
for i in range(len_x):
temp = (x[i] - x_avg)
var_x += temp**2
cov_xy += temp*(y[i] - y_avg)
slope = cov_xy / var_x
y_interc = y_avg - slope*x_avg
return (slope, y_interc)
Here's what I think is happening with Numba: 以下是我认为Numba正在发生的事情:
Numba works on Numpy
arrays. Numba在Numpy
阵列上工作。 Nothing else. 没有其他的。 Everything else has nothing to do with Numba
. 其他一切都与Numba
。
zip
returns an iterator of arbitrary items, which Numba cannot see into. zip
返回Numba无法看到的任意项的迭代器。 Thus Numba cannot do much compiling. 因此,Numba无法进行太多编译。
Looping over the indexes with a for i in range(...)
is likely to produce a much better result and allow much stronger type inference. 使用for i in range(...)
循环索引可能会产生更好的结果,并允许更强大的类型推断。
Using the builtin sum() could be causing problems. 使用内置sum()可能会导致问题。
Here's linear regression code that will run faster in Numba: 这里的线性回归代码在Numba中运行得更快:
@numba.jit
def ols(x, y):
"""Simple OLS for two data sets."""
M = x.size
x_sum = 0.
y_sum = 0.
x_sq_sum = 0.
x_y_sum = 0.
for i in range(M):
x_sum += x[i]
y_sum += y[i]
x_sq_sum += x[i] ** 2
x_y_sum += x[i] * y[i]
slope = (M * x_y_sum - x_sum * y_sum) / (M * x_sq_sum - x_sum**2)
intercept = (y_sum - slope * x_sum) / M
return slope, intercept
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.