Tensorflow: How to train a Linear Regression?

Question

I'm learning to train a Linear Regression model via TensorFlow. It's quite a simple formula:

y = W * x + b

I have generated a sample data:

After the model training I can see in Tensorboard that " W " is correct when " b " goes a completely wrong way. So, Loss is quite high.

Here is my code.

QUESTION

Why is " b " being trained a wrong way?

Shall I do something with the optimizer?

Answer 1

On line 16 , you are adding gaussian noise with a standard deviation of 300!!

noise = np.random.normal(scale=n, size=(N, 1))

Try using:

noise = np.random.normal(size=(N, 1))

That's using mean=0 and std=1 (standard Gaussian noise).

Also, 20k iterations is more than enough (in this problem) for training.

For a more comprehensive explanation of what is happening, look at your plot. Given an x value, the possible values for y have thousands of units of difference. That means that there are a lot of lines that explain your data. Hence a lot of values for B are possible, but no matter which one you choose (even the true b value) all of them are going to have a big loss.

Answer 2

优化工作正常，但问题出在参数b其估计受noise的初始“骰子滚动”（标准偏差为N ）的影响远大于b_true的实际值（后者小得多）比N ）。