Linear Regression with python - gradient descent error

Question

I have been trying to implement my own Linear Regression from scratch using python but have been facing a issue during the last days.

import pandas as pd
import numpy as np
from sklearn.datasets import load_boston
import matplotlib.pyplot as plt

Answer 1

Your cost function is wrong, it should be:

 cost = 1/(2*m) * np.sum(np.power(error,2)) 

Also, try to initialize your weights as random values between 0 an 1 and scale your inputs to range 0-1.

Answer 2

我有一个相同的问题，我通过标准化x值解决了。

Answer 3

I think that you are making a mistake in the gradient descent algorithm. When updating the values for "W" vector it should be:

W = W - (learning_rate/m) * derivate.sum()

Answer 4

The learning rate is too large. I try learning_rate = 0.000001, and it converges normally.