![](/img/trans.png)
[英]Machine Learning: Why is Linear Regression Cost Function First Decreasing then Increasing?
[英]Increasing cost for linear regression
為了訓練目的,我在python中實現了線性回歸。 問題是成本增加而不是減少。 對於數據,我使用翼型自噪聲數據集。 數據可以在這里找到
我導入數據如下:
import pandas as pd
def features():
features = pd.read_csv("data/airfoil_self_noise/airfoil_self_noise.dat.txt", sep="\t", header=None)
X = features.iloc[:, 0:5]
Y = features.iloc[:, 5]
return X.values, Y.values.reshape(Y.shape[0], 1)
我的線性回歸代碼如下:
import numpy as np
import random
class linearRegression():
def __init__(self, learning_rate=0.01, max_iter=20):
"""
Initialize the hyperparameters of the linear regression.
:param learning_rate: the learning rate
:param max_iter: the max numer of iteration to perform
"""
self.lr = learning_rate
self.max_iter = max_iter
self.m = None
self.weights = None
self.bias = None
def fit(self, X, Y):
"""
Run gradient descent algorithm
:param X: the inputs
:param Y: the outputs
:return:
"""
self.m = X.shape[0]
self.weights = np.random.normal(0, 0.1, (X.shape[1], 1))
self.bias = random.normalvariate(0, 0.1)
for iter in range(0, self.max_iter):
A = self.__forward(X)
dw, db = self.__backward(A, X, Y)
J = (1/(2 * self.m)) * np.sum(np.power((A - Y), 2))
print("at iteration %s cost is %s" % (iter, J))
self.weights = self.weights - self.lr * dw
self.bias = self.bias - self.lr * db
def predict(self, X):
"""
Make prediction on the inputs
:param X: the inputs
:return:
"""
Y_pred = self.__forward(X)
return Y_pred
def __forward(self, X):
"""
Compute the linear function on the inputs
:param X: the inputs
:return:
A: the activation
"""
A = np.dot(X, self.weights) + self.bias
return A
def __backward(self, A, X, Y):
"""
:param A: the activation
:param X: the inputs
:param Y: the outputs
:return:
dw: the gradient for the weights
db: the gradient for the bias
"""
dw = (1 / self.m) * np.dot(X.T, (A - Y))
db = (1 / self.m) * np.sum(A - Y)
return dw, db
然后我實例化linearRegression類如下:
X, Y = features()
model = linearRegression()
X_train, X_test, y_train, y_test = train_test_split(X, Y, test_size=0.33, random_state=42)
model.fit(X_train, y_train)
我試圖找出為什么成本在增加但到目前為止我無法找出原因。 如果有人能指出我正確的方向,我們將不勝感激。
通常,如果您選擇較大的學習率,您可能會遇到類似的問題。 我試圖檢查你的代碼,我的意見是:
你的學習速度太高 。 當我使用未經修改的代碼運行時,學習率為1e-7而不是0.01,我可以可靠地降低成本。
通常,當成本增加時,學習率太高。
聲明:本站的技術帖子網頁,遵循CC BY-SA 4.0協議,如果您需要轉載,請注明本站網址或者原文地址。任何問題請咨詢:yoyou2525@163.com.