Solving nonlinear simultaneous equations using `minimize` in Python

Question

I want to solve two simultaneous equations using the scipy.optimize.minimize function in Python, specifically with the dog-leg trust-region algorithm. This requires me to specify the Jacobian of the problem by using scipy.optimize.approx_fprime , as suggested in one solution to my other post .

My MWE is:

import numpy as np
from scipy.integrate import quad
from scipy.optimize import minimize,approx_fprime

def myfunc(guess,a,b,c,d):

    # initial guesses
    x0 = guess[0]
    x1 = guess[1]

    # functions
    z0 = lambda x: c*np.sqrt(a**3*x0)*np.sin(x)/x0**b
    z1 = lambda x: np.cos(b*x0*x)/x1**a

    # numerical integration with 'quad'
    z0int = quad(z0,-200,200,epsabs=1e-8,epsrel=1e-6)[0] - 3.2*d
    z1int = quad(z1,-200,200,epsabs=1e-8,epsrel=1e-6)[0] + c

    return (z0int,z1int)

# constants
a = 0.2
b = 1.1
c = 2.5
d = 0.9

guess = np.array([0.3,0.02]) # initial guesses

myJac = approx_fprime(guess,myfunc,1e-9,a,b,c,d) # Jacobian

# minimisation, want to find x0 such that z0int=0 and z1int=0
xopt = minimize(myfunc,guess,args=(a,b,c,d),method='dogleg',jac=myJac)

print(xopt)

However I get an error TypeError: unsupported operand type(s) for -: 'tuple' and 'tuple' . I'm not really familiar with the Python optimization functions so could you please explain what is wrong and how to correct the code?

Answer 1

For minimisation, your function should return a single integer. You are returning a tuple, so that is the problem. The minimize function checks if the new value is lower then the old one (thus subtract) but it wants to subtract tuples instead of ints.

Change your code to only return a single integer from the function you wish to minimize

EDIT according to comments

def myfunc(guess,a,b,c,d):

    # initial guesses
    x0 = guess[0]
    x1 = guess[1]

    # functions
    z0 = lambda x: c*np.sqrt(a**3*x0)/x0**b
    z1 = lambda x: np.cos(b*x0)/x1**a

    # numerical integration with 'quad'
    z0int = quad(z0,-200,200,epsabs=1e-8,epsrel=1e-6)[0] - 3.2*d
    z1int = quad(z1,-200,200,epsabs=1e-8,epsrel=1e-6)[0] + c

    return (z0int,z1int) # <-- This should only return 1 single integer

Answer 2

For solving a system of equations, you are aiming to minimize the sum of squares of left-hand sides. For this, you might be better off using least_squares instead of more general minimize . There is an example in the documentation, https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.least_squares.html

Under the hood it uses a trust-region type approach.

Answer 3

You can rewrite the function to yield both elements, one at a time, instead of returning a tuple. One solution will be returned the 1st time, 3rd time, ..., odd times; the other solution returned every even time. You can then write a new function that you minimize instead; this new function will initialize 2 lists (evens + odds), using every other yielded element for each list. This new function will return some type of error metric with respect to both solutions such that its minimization yields the best 2 solutions.