Optimizing a Nested loop that contains 8 for loops to minimize a function

Question

I have a python code that maximizes a function over 8 parameters using a nested for loop. It takes approximately 16 Minutes to execute which is way too much because I have to do the optimization numerous time for the problem I am trying to solve.

I tried:

1.) Replacing the for loop by list comprehension but there was no change in performance.

2.) Jug to parallize but the entire system freezes and restarts.

My Questions:

1.) Is there any other way to parallelize a nested for loop using multiprocessing module?

2.) Is there any way I can replace the nested loop with completely different method to maximize the function?

Code Snippet: 
def SvetMaxmization(): #Maximization function
    Max = 0
    res = 1.0 # Step Size, execution time grows expo if the value is reduced
    for a1 in np.arange(0, pi, res):
        for a2 in np.arange(0,pi, res):
            for b1 in np.arange(0,pi, res):
                for b2 in np.arange(0,pi, res):
                    for c1 in np.arange(0,pi, res): 
                        for c2 in np.arange(0,pi, res):
                            for d1 in np.arange(0,pi, res):
                                for d2 in np.arange(0,pi, res):
                                   present =Svet(a1,a2,b1,b2,c1,c2,d1,d2) #function to be maximized 
                                   if present > Max:
                                        Max = present

svet() function:

def Svet(a1,a2,b1,b2,c1,c2,d1,d2):
    Rho = Desnitystate(3,1) #Rho is a a matrix of dimension 4x4
    CHSH1 = tensor(S(a1),S(b1)) + tensor(S(a1),S(b2)) + tensor(S(a2),S(b1)) - tensor(S(a2),S(b2)) # S returns a matrix of dimension 2x2
    CHSH2 = tensor(S(a1),S(b1)) + tensor(S(a1),S(b2)) + tensor(S(a2),S(b1)) - tensor(S(a2),S(b2))
    SVet3x1 = tensor(CHSH1, S(c2)) + tensor(CHSH2, S(c1))
    SVet3x2 = tensor(CHSH2, S(c1)) + tensor(CHSH1, S(c2))                   
    SVet4x1 = tensor(SVet3x1, S(d2)) + tensor(SVet3x2, S(d1))           
    Svd = abs((SVet4x1*Rho).tr())

    return Svd

System details: Intel Core I5 clocked at 3.2GHz

Thanks for your time!!

Answer 1

It's hard to give a single "right" answer, as it will depend a lot on the behaviour of your cost function.

But, considering that you are now doing a gridsearch over the parameter space (basically brute-forcing the solution), I think there are some things worth trying.

1. See if you can use a more sophisticated optimization algorithm. See the scipy.optimize module , eg just if

   x0 = ... # something bounds = [(0,np.pi) for _ in range(len(x0))] result = minimize(Svet, x0, bounds=bounds)

   can solve the problem.

2. If the cost function is so badly behaved that none of those methods work, your only hope is probably to speed up the execution of the cost function itself. In my own experience, I would try the following:

   1. numba is a good first alternative because it is very simple to try since it does not require you to change anything in your current code. It doesn't always speed up your code though.

   2. Rewrite the cost function with Cython . This requires some work on your part, but will likely give a large boost in speed. Again, this depends on the nature of your cost function.

   3. Rewrite using eg C, C++, or any other "fast" language.