Python 中梯度下降算法的轮廓图

Question

I'm trying to apply gradient descent to a simple linear regression model, when plotting a 2D graph I get the intended result but when I switch into a contour plot I don't the intended plot, I would like to know where my mistake is.

Here is the code:

def J(b_0, b_1, x, y):
  return (1/len(y))*(y - b_0 - b_1*x)**2

def dJ_1(b_0, b_1, x, y):
  return (2/len(y))*np.sum(x*(b_1*x + b_0 - y))

def dJ_0(b_0, b_1, x, y):
  return (2/100)*np.sum((b_1*x + b_0 - y))

x = np.linspace(-1, 1, 100)
y = np.linspace(-2, 2, 100)

b_0 = 5
b_1 = 5
parameters_0 = [b_0]
parameters_1 = [b_1]
for i in range(99):
  b_1 -= 0.1*dJ_1(b_0, b_1, x, y)
  b_0 -= 0.1*dJ_0(b_0, b_1, x, y)
  parameters_0.append(b_0)
  parameters_1.append(b_1)

plt.figure(figsize=(4, 8))
plt.plot(np.linspace(-2, 7, 100), J(np.linspace(-2, 7, 100), parameters_1[-1], -1, -2))
plt.plot(np.array(parameters_0), J(np.array(parameters_0), parameters_1[-1], -1, -2), color="C1")
plt.plot(np.array(parameters_0), J(np.array(parameters_0), parameters_1[-1], -1, -2), '-o', color="C1")
plt.xlabel(r"$\beta_0$")
plt.ylabel(r"$J(\beta_0)$")
plt.show()

The first plot:

plt.figure(figsize=(4, 8))
plt.plot(np.linspace(-4, 7, 100), J(parameters_0[-1], np.linspace(-4, 7, 100), -1, -2))
plt.plot(np.array(parameters_1), J(parameters_0[-1], np.array(parameters_1), -1, -2), color="C1")
plt.plot(np.array(parameters_1), J(parameters_0[-1], np.array(parameters_1), -1, -2), '-o', color="C1")
plt.xlabel(r"$\beta_1$")
plt.ylabel(r"$J(\beta_1)$")
plt.show()

The second plot:

b_0 = np.linspace(-10, 10, 100)
b_1 = np.linspace(-10, 10, 100)
X, Y = np.meshgrid(b_0, b_1)
Z = J(X, Y, x=-1, y=-2)
fig,ax=plt.subplots(1,1)
cp = ax.contourf(X, Y, Z)
fig.colorbar(cp) 
ax.set_xlabel(r"$\beta_0$")
ax.set_ylabel(r"$\beta_1$")
plt.show()

The contour plot is:

Why do I get the above plot rather than a plot similar to this below one for example when the global minima of the cost function is at (0, 2)? Thanks in advance.

Answer 1

Well I think there's no mistake there, you can see from the 2d plot that your gradient descent plot is a quadratic function, thus the way you see it from the contour is as if you see it from the sky to the valley. As to why it doesn't look like a circle, well it's because it's just a 3d quadratic function. I also once made something similar, and the gradient descent plot is just as what you plot. Check it out here at the end of the page