如何停止scipy.optimize.fmin函數並使用優化歷史記錄再次啟動它？

Question

對於需要很長時間（數十分鍾，幾小時，幾天或幾個月）的優化，如果程序出現故障或斷電，則必須能夠使用優化歷史記錄再次啟動優化，但是fmin算法（以及其他幾個）不提供歷史記錄作為輸出或輸入。 如何保存和使用fmin優化歷史記錄的結果，以確保不會損失所有計算投資？

我昨天早上有這個問題，在網上找不到任何答案，所以我整理了自己的解決方案。 見下文。

Answer 1

基本上，實時監控，歷史記錄和恢復fmin的答案是使用可調用函數將函數的輸入和輸出存儲到查找表中。 這是完成的過程：

import numpy as np
import scipy as sp
import scipy.optimize

要存儲歷史記錄，請為輸入創建全局歷史矢量，並為給定輸入的目標函數值創建全局歷史矢量。 我也在這里初始化了初始輸入向量：

x0 = np.array([1.05,0.95])
x_history = np.array([[1e8,1e8]])
fx_history = np.array([[1e8]])

我正在優化Rosenbrock的函數，因為它是典型的優化算法：

def rosen(x):
    """The Rosenbrock function"""
    return sum(100.0*(x[1:]-x[:-1]**2.0)**2.0 + (1-x[:-1])**2.0)

為要優化的函數創建包裝器，並在從優化函數請求新輸入並計算目標函數值時寫入全局變量。 對於每次迭代，搜索歷史記錄以查找是否先前已計算出所請求向量的目標函數值，如果已經計算過，則使用先前計算出的值。 為了“模擬”電源故障，我創建了一個名為powerFailure的變量，以在優化完成收斂之前結束優化。 然后我關閉powerFailure以查看優化完成。

def f(x):
    global firstPoint, iteration, x_history, fx_history

    iteration = iteration + 1
    powerFailure = True
    failedIteration = 10
    previousPoint = False
    eps = 1e-12

    if powerFailure == True:
        if iteration == failedIteration:
            raise Exception("Optimization Ended Early Due to Power Failure")

    for i in range(len(x_history)):
        if abs(x_history[i,0]-x[0])<eps and abs(x_history[i,1]-x[1])<eps:
            previousPoint = True
            firstPoint=False
            fx = fx_history[i,0]
            print "%d: f(%f,%f)=%f (using history)" % (iteration,x[0],x[1],fx)

    if previousPoint == False:
        fx = rosen(x)
        print "%d: f(%f,%f)=%f" % (iteration,x[0],x[1],fx)

    if firstPoint == True:
        x_history = np.atleast_2d([x])
        fx_history = np.atleast_2d([fx])
        firstPoint = False
    else:
        x_history = np.concatenate((x_history,np.atleast_2d(x)),axis=0)  
        fx_history = np.concatenate((fx_history,np.atleast_2d(fx)),axis=0)

    return fx

最后，我們運行優化。

firstPoint = True
iteration = 0

xopt, fopt, iter, funcalls, warnflag, allvecs = sp.optimize.fmin(f,x0,full_output=True,xtol=0.9,retall=True)

發生電源故障后，優化將在迭代9處結束。“重新打開電源”后，該函數將打印

1: f(1.050000,0.950000)=2.328125 (using history)
2: f(1.102500,0.950000)=7.059863 (using history)
3: f(1.050000,0.997500)=1.105000 (using history)
4: f(0.997500,0.997500)=0.000628 (using history)
5: f(0.945000,1.021250)=1.647190 (using history)
6: f(0.997500,1.045000)=0.249944 (using history)
7: f(0.945000,1.045000)=2.312665 (using history)
8: f(1.023750,1.009375)=0.150248 (using history)
9: f(1.023750,0.961875)=0.743420 (using history)
10: f(1.004063,1.024219)=0.025864
11: f(0.977813,1.012344)=0.316634
12: f(1.012266,1.010117)=0.021363
13: f(1.005703,0.983398)=0.078659
14: f(1.004473,1.014014)=0.002569
15: f(0.989707,1.001396)=0.047964
16: f(1.006626,1.007937)=0.002916
17: f(0.995347,1.003577)=0.016564
18: f(1.003806,1.006847)=0.000075
19: f(0.996833,0.990333)=0.001128
20: f(0.998743,0.996253)=0.000154
21: f(1.005049,1.005600)=0.002072
22: f(0.999387,0.999525)=0.000057
Optimization terminated successfully.
         Current function value: 0.000057
         Iterations: 11
         Function evaluations: 22