Scipy: difference between optimize.fmin and optimize.leastsq

Question

What's the difference between scipy's optimize.fmin and optimize.leastsq? They seem to be used in pretty much the same way in this example page . The only difference I can see is that leastsq actually calculates the sum of squares on its own (as its name would suggest) while when using fmin one has to do this manually. Other than that, are the two functions equivalent?

Answer 1

Different algorithms underneath.

fmin is using the simplex method; leastsq is using least squares fitting.

Answer 2

Just to add some information, I am developing a module to fit a biexponential function and the time difference between leastsq and minimize seems to be almost 100 times. Have a look at the code below for more details.

I used a biexponential curve which is a sum of two exponents and the model function has 4 parameters to fit. S, f, D_star and D.

All default parameters for fitting were used

S [fe^(-x * D_star) + (1 - f) e^(-x * D)]

('Time taken for minimize:', 0.011617898941040039)
('Time taken for leastsq :', 0.0003180503845214844)

The code used:

import numpy as np
from scipy.optimize import minimize, leastsq
from time import time


def ivim_function(params, bvals):
    """The Intravoxel incoherent motion (IVIM) model function.

        S(b) = S_0[f*e^{(-b*D\*)} + (1-f)e^{(-b*D)}]

        S_0, f, D\* and D are the IVIM parameters.

    Parameters
    ----------
        params : array
                parameters S0, f, D_star and D of the model

        bvals : array
                bvalues

    References
    ----------
    .. [1] Le Bihan, Denis, et al. "Separation of diffusion
               and perfusion in intravoxel incoherent motion MR
               imaging." Radiology 168.2 (1988): 497-505.
    .. [2] Federau, Christian, et al. "Quantitative measurement
               of brain perfusion with intravoxel incoherent motion
               MR imaging." Radiology 265.3 (2012): 874-881.
    """
    S0, f, D_star, D = params
    S = S0 * (f * np.exp(-bvals * D_star) + (1 - f) * np.exp(-bvals * D))
    return S


def _ivim_error(params, bvals, signal):
    """Error function to be used in fitting the IVIM model
    """
    return (signal - ivim_function(params, bvals))


def sum_sq(params, bvals, signal):
    """Sum of squares of the errors. This function is minimized"""
    return np.sum(_ivim_error(params, bvals, signal)**2)

x0 = np.array([100., 0.20, 0.008, 0.0009])
bvals = np.array([0., 10., 20., 30., 40., 60., 80., 100.,
                  120., 140., 160., 180., 200., 220., 240.,
                  260., 280., 300., 350., 400., 500., 600.,
                  700., 800., 900., 1000.])
data = ivim_function(x0, bvals)

optstart = time()
opt = minimize(sum_sq, x0, args=(bvals, data))
optend = time()
time_taken = optend - optstart
print("Time taken for opt:", time_taken)


lstart = time()
lst = leastsq(_ivim_error,
              x0,
              args=(bvals, data),)
lend = time()
time_taken = lend - lstart
print("Time taken for leastsq :", time_taken)

print('Parameters estimated using minimize :', opt.x)
print('Parameters estimated using leastsq :', lst[0])