Why does “scipy.optimize.minimize” gives me such a bad fit?

Question

I have a function y(x,z) with two variables x , z and 6 coefficients a , b , c , d , e , f . I have the data for x , z and let's say for testing purpose the data of the coefficients. With those datas I calculate my y .

Then I want to fit the function with the datas of x , z and the calculated y to get the coefficients and compare them with the testing purpose one.

import numpy as np
from scipy.optimize import minimize

x = np.array([0,0.25,0.5,0.75,1]) # data of variable x
z = np.array([0,0.25,0.5,0.75,1]) # data of variable z

def func(pars,x,z): #my function
    a,b,c,d,e,f = pars
    return a*x**2+b*x+c+d*z+e*z*x+f*z*x**2

a = np.array([1,1,1,1,1])  #define coefficients to get the y data and compare them later with fit
b = np.array([0.5,0.5,0.5,0.5,0.5])
c = np.array([0.25,0.25,0.25,0.25,0.25])
d = np.array([1,1,1,1,1])
e = np.array([0.5,0.5,0.5,0.5,0.5])
f = np.array([0.25,0.25,0.25,0.25,0.25])

y = []
y.append(func((a,b,c,d,e,f),x,z)) #calculate the y data
print(y)

def resid(pars,x,z,y): #residual function
    return ((func(pars,x,z) - y) ** 2).sum()

pars0 = np.array([0,0,0,0,0,0])
res = minimize(resid, pars0,args=(x,z,y), method='cobyla',options={'maxiter': 5000000})
print("a = %f , b = %f, c = %f, d = %f, e = %f, f = %f" % (res.x[0], res.x[1], res.x[2], res.x[3], res.x[4], res.x[5]))

I am getting the following coefficients from fitting:

a = 1.181149 , b = 1.228558, c = 0.253053, d = 0.219143, e = 0.444941, f = 0.172369

Compared with my coefficients for calculating the y data the fitting is not realy what I would call adquate. Can someone explain me why my fit is so bad?

PS: If someone is wondering, I use cobyla because I have to define some constraints later on. This is just a testing code to find out where my problem is located (hopefully).

Answer 1

Looking at res.fun , which is in your case around 1e-5 the fit is actually quite good.

Most likely you found a local mininum of your objective function. To better understand this behaviour, try the code below. This will generate different results for different startpoints. As you will see, you are minimizing, just not to the global minimum. To optimize globally, you have to use other approaches/methods. You could also increase the criteria for when to stop the optimization. Or use a hybrid approach and start at different initial points, solve the local minimization and take the best value.

for i in range(10):
    pars0 = np.random.rand(6) * 1
    res = minimize(resid, pars0, args=(x,z,y), method='cobyla',options={'maxiter': 5000000})
    print("a = %f , b = %f, c = %f, d = %f, e = %f, f = %f" % (res.x[0], res.x[1], res.x[2], res.x[3], res.x[4], res.x[5]))
    print(res.fun)

Try an inital point close to the solution you are seeking. This will most probably yield the global result. If you don't know the vague location of your solution, you may have to use a hybrid/global approach to the minimization.

For example, the initial point:

pars0 = np.array([1,0.5,0.25,1,0.5,0.25]) + np.random.rand(6)*0.01

yields a quite fitting solution.