如何使用fmin_ncg計算成本和theta

Question

我正在Coursera上學習Andrew NG課程，我想在python上實現相同的邏輯。 我正在嘗試計算成本和θ

scipy.optimize.fmin_ncg

這是一個代碼

import numpy as np

from scipy.optimize import fmin_ncg


def sigmoid(z):
    return (1 / (1 + np.exp(-z))).reshape(-1, 1)


def compute_cost(theta, X, y):
    m = len(y)
    hypothesis = sigmoid(np.dot(X, theta))
    cost = (1 / m) * np.sum(np.dot(-y.T, (np.log(hypothesis))) - np.dot((1 - y.T), np.log(1 - hypothesis)))
    return cost


def compute_gradient(theta, X, y):
    m = len(y)
    hypothesis = sigmoid(np.dot(X, theta))
    gradient = (1 / m) * np.dot(X.T, (hypothesis - y))
    return gradient


def main():
    data = np.loadtxt("data/data1.txt", delimiter=",")  # 100, 3

    X = data[:, 0:2]
    y = data[:, 2:]
    m, n = X.shape

    initial_theta = np.zeros((n + 1, 1))
    X = np.column_stack((np.ones(m), X))
    mr = fmin_ncg(compute_cost, initial_theta, compute_gradient, args=(X, y), full_output=True)
    print(mr)

if __name__ == "__main__":
    main()

當我嘗試運行此程序時，出現如下所示的錯誤和異常

Traceback (most recent call last):
  File "/file/path/without_regression.py", line 78, in <module>
    main()
  File "/file/path/without_regression.py", line 66, in main
    mr = fmin_ncg(compute_cost, initial_theta, compute_gradient, args=(X, y), full_output=True)
  File "/usr/local/anaconda3/envs/ml/lib/python3.6/site-packages/scipy/optimize/optimize.py", line 1400, in fmin_ncg
    callback=callback, **opts)
  File "/usr/local/anaconda3/envs/ml/lib/python3.6/site-packages/scipy/optimize/optimize.py", line 1497, in _minimize_newtoncg
    dri0 = numpy.dot(ri, ri)
ValueError: shapes (3,1) and (3,1) not aligned: 1 (dim 1) != 3 (dim 0)

我不明白這個錯誤。 可能是因為我是初學者，這對我來說並不冗長。

如何使用scipy.optimize.fmin_ncg或其他任何最小化技術（例如scipy.optimize.minimize(...)來計算成本和theta？

Answer 1

如評論中所述：

暫時不參考文檔，應該始終使用一維數組。

相關的SO問題

import numpy as np
a = np.random.random(size=(3,1))   # NOT TO USE!
a.shape  # (3, 1)
a.ndim   # 2
b = np.random.random(size=3)       # TO USE!
b.shape  # (3,)                    
b.ndim   # 1

這適用於您的x0 （如果不使用python-lists）和漸變。

快速的技巧（=在漸變中降低暗淡程度），例如：

gradient = (1 / m) * np.dot(X.T, (hypothesis - y)).ravel()  # .ravel()!
...      
initial_theta = np.zeros(n + 1)  # drop extra-dim

使代碼運行：

Optimization terminated successfully.
         Current function value: 0.203498
         Iterations: 27
         Function evaluations: 71
         Gradient evaluations: 229
         Hessian evaluations: 0
(array([-25.13045417,   0.20598475,   0.2012217 ]), 0.2034978435366513, 71, 229, 0, 0)

額外：在調試期間，我還檢查了梯度本身對數值微分的計算（推薦！），使用x0看起來不錯：

from scipy.optimize import check_grad as cg
print(cg(compute_cost, compute_gradient, initial_theta, X, y))
# 1.24034933954e-05