numpy.polyfit与scipy.odr

Question

I have a data set which in theory is described by a polynomial of the second degree. I would like to fit this data and I have used numpy.polyfit to do this. However, the down side is that the error on the returned coefficients is not available. Therefore I decided to also fit the data using scipy.odr . The weird thing was that the coefficients for the polynomial deviated from each other.

I do not understand this and therefore decided to test both fitting routines on a set of data that I produce my self:

import numpy
import scipy.odr
import matplotlib.pyplot as plt

x = numpy.arange(-20, 20, 0.1)
y = 1.8 * x**2 -2.1 * x + 0.6 + numpy.random.normal(scale = 100, size = len(x))

#Define function for scipy.odr
def fit_func(p, t):
  return p[0] * t**2 + p[1] * t + p[2]

#Fit the data using numpy.polyfit
fit_np = numpy.polyfit(x, y, 2)

#Fit the data using scipy.odr
Model = scipy.odr.Model(fit_func)
Data = scipy.odr.RealData(x, y)
Odr = scipy.odr.ODR(Data, Model, [1.5, -2, 1], maxit = 10000)
output = Odr.run()
#output.pprint()
beta = output.beta
betastd = output.sd_beta

print "poly", fit_np
print "ODR", beta

plt.plot(x, y, "bo")
plt.plot(x, numpy.polyval(fit_np, x), "r--", lw = 2)
plt.plot(x, fit_func(beta, x), "g--", lw = 2)

plt.tight_layout()

plt.show()

An example of an outcome is as follows:

poly [ 1.77992643 -2.42753714  3.86331152]
ODR [   3.8161735   -23.08952492 -146.76214989]

In the included image, the solution of numpy.polyfit (red dashed line) corresponds pretty well. The solution of scipy.odr (green dashed line) is basically completely off. I do have to note that the difference between numpy.polyfit and scipy.odr was less in the actual data set I wanted to fit. However, I do not understand where the difference between the two comes from, why in my own testing example the difference is extremely big, and which fitting routine is better?

I hope you can provide answers that might help me give a better understanding between the two fitting routines and in the process provide answers to the questions I have.

Answer 1

In the way you are using ODR it does a full orthogonal distance regression. To have it do a normal nonlinear least squares fit add

Odr.set_job(fit_type=2)

before starting the optimization and you will get what you expected.

The reason that the full ODR fails so badly is due to not specifying weights/standard deviations. Obviously it does hard to interpret that point cloud and assumes equal wheights for x and y. If you provide estimated standard deviations, odr will yield a good (though different of course) result, too.

Data = scipy.odr.RealData(x, y, sx=0.1, sy=10)

Answer 2

The actual problem is that the odr output has the beta coefficients in the opposite order than numpy.polyfit has. So the green curve is not calculated correctly. To plot it, use instead

plt.plot(x, fit_func(beta[::-1], x), "g--", lw = 2)