[英]Torch neural network does not train
I have implemented a very simple neural network in the torch
framework我在
torch
框架中实现了一个非常简单的神经网络
def mlp(sizes, activation, output_activation=torch.nn.Identity):
layers = []
for j in range(len(sizes)-1):
act = activation if j < len(sizes)-1 else output_activation
layers += [torch.nn.Linear(sizes[j], sizes[j+1]), act()]
return torch.nn.Sequential(*layers)
In order to train a network to make regression on the function y=sin(x)为了训练网络对函数 y=sin(x) 进行回归
x = torch.linspace(-math.pi, math.pi, 2000, device=device, dtype=dtype)
y = torch.sin(x)
the training code is here培训代码在这里
size = [1,20,20,1]
activation = torch.nn.ReLU
model = mlp(size, activation)
optimizer = torch.optim.SGD(model.parameters(), lr=0.002)
n_epoch = 600
mse_loss = torch.nn.MSELoss()
X = x.unsqueeze(-1)
for i in range(n_epoch):
y_pred = model(X)
step_loss = mse_loss(y_pred, y)
optimizer.zero_grad()
step_loss.backward()
optimizer.step()
Unfortunately, the network only learn an almost constant function $y=0$.不幸的是,网络只学习了一个几乎恒定的函数 $y=0$。 I have already tried many things
我已经尝试了很多东西
But nothing seems to work.但似乎没有任何效果。 The problem is so simple that I think there is an error in the code.
问题很简单,我认为代码中有错误。
I am not sure if this is the main cause, but the statement我不确定这是否是主要原因,但声明
act = activation if j < len(sizes)-1 else output_activation
appears to be logically incorrect.似乎逻辑不正确。 In the loop,
j
can take values from 0
to len(sizes)-1
, so the condition is always true.在循环中,
j
可以取从0
到len(sizes)-1
的值,因此条件始终为真。 This means that your network has a ReLU right at the end, and so can only ever give non-negative outputs.这意味着你的网络最后有一个 ReLU,因此只能给出非负输出。 This can be corrected by changing that statement to:
这可以通过将该语句更改为:
act = activation if j < len(sizes)-2 else output_activation
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.