Why is scipy.optimize.minimize trying to pass in weird arguments to my objective function?

Question

I have a class that helps instantiate a statistical model. Some of its data members are parameters. I am trying to write a method that optimizes these parameters. The objection function is based on the negative of the likelihood function. This likelihood function is implemented itself as a class method, which uses the values of the class data members in its calculation.

I know this is bad style, but every time the objective function gets called by scipy.optimize.minimize() , it changes the objects data members to the better ones. I am less concerned with why this is bad, and more concerned with why this isn't working. Below the code is the full traceback.

It seems like it works partially. It runs for a few seconds on test data, but then it triggers my assertion. It seems that minimize() does something weird when it's nearing the end of its optimizations. Why would it try to pass in different types of arguments to the objective function obj_fun() ? In my IPython interpreter I check the object's parameters afterwards, and it seems to be arriving at the expected result.

I tried looking through some of the source of scipy. It's very confusing, though. There is a lot of ambiguous variable naming and function wrapping. Can anybody give me some color on why this is happening, and how to fix it? Again I would like to keep this optimization stuff inside my class.

class MyThing(object):
    .
    .
    .
    def mle_fit(self, y, inpt, optim_these):
        #step 1: figure out what you want to optimize
        self.optimize_these = optim_these
        #step 2: get inital flat parameter vector 
        self._make_list_optimizable_hyp_pars()
        init_guess = self.flat_hyper_params
        #step 3: run minimize
        def obj_fun(pars):
            # returns negative log likelihood 
            assert len(pars) == len(init_guess) # HERE #
            self.flat_hyper_params = pars
            self._unflatten_new_hps()
            self.like(y, inpt)
            return .5 * self.neg_2_log_like

        res = minimize(obj_fun, init_guess, method = 'BFGS')

Traceback:

  File "/home/taylor/anaconda/lib/python2.7/site-packages/scipy/optimize/_minimize.py", line 419, in minimize
    return _minimize_bfgs(fun, x0, args, jac, callback, **options)

  File "/home/taylor/anaconda/lib/python2.7/site-packages/scipy/optimize/optimize.py", line 850, in _minimize_bfgs
    old_fval, old_old_fval)

  File "/home/taylor/anaconda/lib/python2.7/site-packages/scipy/optimize/optimize.py", line 690, in _line_search_wolfe12
    old_fval, old_old_fval)

  File "/home/taylor/anaconda/lib/python2.7/site-packages/scipy/optimize/linesearch.py", line 263, in line_search_wolfe2
    derphi0, c1, c2, amax)

  File "/home/taylor/anaconda/lib/python2.7/site-packages/scipy/optimize/linesearch.py", line 363, in scalar_search_wolfe2
    phi0, derphi0, c1, c2)

  File "/home/taylor/anaconda/lib/python2.7/site-packages/scipy/optimize/linesearch.py", line 498, in _zoom
    phi_aj = phi(a_j)

  File "/home/taylor/anaconda/lib/python2.7/site-packages/scipy/optimize/linesearch.py", line 239, in phi
    return f(xk + alpha * pk, *args)

  File "/home/taylor/anaconda/lib/python2.7/site-packages/scipy/optimize/optimize.py", line 281, in function_wrapper
    return function(*(wrapper_args + args))

  File "MyThing.py", line 222, in obj_fun
    assert len(pars) == len(init_guess)

Answer 1

If I remember correctly, SciPy, when optimizing, can send both scalars and arrays to the function to be minimized. Thus, if you optimize f(x) , you will get, say, x = 3.14 and x = array([1, 4]) . (This would be for speeding up calculations, for objective functions f that use NumPy array functions.) If this is indeed the case, having your code handle this would solve the problem.

Now, you can easily check what the situation is with a check like print pars, type(pars), len(pars) in obj_fun() .