Cython/numpy vs pure numpy for least square fitting

Question

A TA at school showed me this code as an example of a least square fitting algorithm.

import numpy as np
#return the coefficients (a0,..aN) of the fit y=a0+a1*x+..an*x^n
#with associated sigma dy
#x,y,dy are all np.arrays with dtype= np.float64
def fit_poly(x,y,dy,n):
  V = np.asmatrix(np.diag(dy**2))
  M = []

  for k in range(n+1):
      M.append(x**k)
  M = np.asmatrix(M).T
  theta = (M.T*V.I*M).I*M.T*V.I*np.asmatrix(y).T
  cov_t = (M.T*V.I*M).I

  return np.asarray(theta.T)[0], np.asarray(cov_t)

Im trying to optimize his codes using cython. i got this code

cimport numpy as np
import numpy as np
cimport cython

@cython.boundscheck(False)
@cython.wraparound(False) 
cpdef poly_c(np.ndarray[np.float64_t, ndim=1] x ,
   np.ndarray[np.float64_t, ndim=1] y np.ndarray[np.float64_t,ndim=1]dy , np.int n):

   cdef np.ndarray[np.float64_t, ndim=2] V, M

   V=np.asmatrix(np.diag(dy**2),dtype=np.float64)
   M=np.asmatrix([x**k for k in range(n+1)],dtype=np.float64).T

   return ((M.T*V.I*M).I*M.T*V.I*(np.asmatrix(y).T))[0],(M.T*V.I*M).I

But the runtime seems to be the same for both programs,i did used an 'assert' to make sure the outputs where the same. What am i missing/doing wrong?

Thank you for your time and hopefully you can help me.

ps: this is the code im profiling with(not sure if i can call this profiling but w/e)

import numpy as np
from polyC import poly_c
from time import time
from pancho_fit import fit_poly
#pancho's the T.A,sup pancho
x=np.arange(1,1000)
x=np.asarray(x,dtype=np.float64)
y=3*x+np.random.random(999)
y=np.asarray(y,dtype=np.float64)
dy=np.array([y.std() for i in range(1,1000)],dtype=np.float64)
t0=time()
a,b=poly_c(x,y,dy,4)
#a,b=fit_poly(x,y,dy,4)
print("time={}s".format(time()-t0))

Answer 1

Except for [x**k for k in range(n+1)] I don't see any iterations for cython to improve. Most of the action is in matrix products. Those are already done with compiled code (with np.dot for ndarrays ).

And n is only 4, not many iterations.

But why iterate this?

In [24]: x=np.arange(1,1000.)
In [25]: M1=x[:,None]**np.arange(5)
# np.matrix(M1)

does the same thing.

So no, this does not look like a good cython candidate - not unless you are prepared to write out all those matrix products in compilable detail.

I'd skip also the asmatrix stuff and use regular dot , @ and einsum , but that's more a matter of style than speed.