[英]Unable to figure out inplace operation in the pytorch code?
I have the following implementation in PyTorch for learning using LSTM:我在 PyTorch 中有以下实现用于使用 LSTM 进行学习:
https://gist.github.com/rahulbhadani/f1d64042cc5a80280755cac262aa48aa https://gist.github.com/rahulbhadani/f1d64042cc5a80280755cac262aa48aa
However, the code is experiencing in-place operation error但是,代码遇到就地操作错误
where my error output is:我的错误输出是:
/home/ivory/anaconda3/lib/python3.7/site-packages/ipykernel_launcher.py:10: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
# Remove the CWD from sys.path while we load stuff.
---------------------------------------------------------------------------
RuntimeError Traceback (most recent call last)
<ipython-input-86-560ec78f2b64> in <module>
27 linear = torch.nn.Linear(hidden_nums, output_dim)
28
---> 29 global_loss_list = global_training(lstm2)
<ipython-input-84-152890a3028c> in global_training(optimizee)
3 adam_global_optimizer = torch.optim.Adam([{'params': optimizee.parameters()},
4 {'params':linear.parameters()}], lr = 0.0001)
----> 5 _, global_loss_1 = learn2(LSTM_Optimizee, training_steps, retain_graph_flag=True, reset_theta=True)
6
7 print(global_loss_1)
<ipython-input-83-0357a528b94d> in learn2(optimizee, unroll_train_steps, retain_graph_flag, reset_theta)
43 # requires_grad=True. These are accumulated into x.grad for every
44 # parameter x. In pseudo-code: x.grad += dloss/dx
---> 45 loss.backward(retain_graph = retain_graph_flag) #The default is False, when the optimized LSTM is set to True
46
47 print('x.grad: {}'.format(x.grad))
~/anaconda3/lib/python3.7/site-packages/torch/tensor.py in backward(self, gradient, retain_graph, create_graph)
116 products. Defaults to ``False``.
117 """
--> 118 torch.autograd.backward(self, gradient, retain_graph, create_graph)
119
120 def register_hook(self, hook):
~/anaconda3/lib/python3.7/site-packages/torch/autograd/__init__.py in backward(tensors, grad_tensors, retain_graph, create_graph, grad_variables)
91 Variable._execution_engine.run_backward(
92 tensors, grad_tensors, retain_graph, create_graph,
---> 93 allow_unreachable=True) # allow_unreachable flag
94
95
RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.FloatTensor [1, 10]] is at version 1; expected version 0 instead. Hint: enable anomaly detection to find the operation that failed to compute its gradient, with torch.autograd.set_detect_anomaly(True).
I have tried to trace the error but didn't succeed.我试图追踪错误,但没有成功。 Any help in this regard will be appreciated.
在这方面的任何帮助将不胜感激。
Thanks.谢谢。
I think the issue is with the following line:我认为问题出在以下几行:
global_loss_list.append(global_loss.detach_())
The convention in PyTorch for in-place operations is using _
at the end of the function name (as in detach_
). PyTorch 中就地操作的约定是在函数名称的末尾使用
_
(如detach_
)。 I believe you shouldn't be detaching in-place.我相信你不应该就地分离。 In other words, change
detach_
to detach
换句话说,将
detach_
更改为detach
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.