[英]Can't fix: RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation
Currently, I'm trying to replicate a DeblurGanV2 network.目前,我正在尝试复制 DeblurGanV2 网络。 At the moment, I'm working on performing the training.
目前,我正在努力进行培训。 Here is my current status of my trainings pipeline:
这是我目前的培训管道状态:
from torch.autograd import Variable
torch.autograd.set_detect_anomaly(mode=True)
total_generator_loss = 0
total_discriminator_loss = 0
psnr_score = 0.0
used_loss_function = 'wgan_gp_loss'
for epoch in range(n_epochs):
#set to train mode
generator.train(); discriminator.train()
tqdm_bar = tqdm(train_loader, desc=f'Training Epoch {epoch} ', total=int(len(train_loader)))
for batch_idx, imgs in enumerate(tqdm_bar):
#load imgs to cpu
blurred_images = imgs["blurred"].cuda()
sharped_images = imgs["sharp"].cuda()
# generator output
deblurred_img = generator(blurred_images)
# denormalize
with torch.no_grad():
denormalized_blurred = denormalize(blurred_images)
denormalized_sharp = denormalize(sharped_images)
denormalized_deblurred = denormalize(deblurred_img)
# get D's output
sharp_discriminator_out = discriminator(sharped_images)
deblurred_discriminator_out = discriminator(deblurred_img)
# set critic_updates
if used_loss_function== 'wgan_gp_loss':
critic_updates = 5
else:
critic_updates = 1
#train discriminator
discriminator_loss = 0
for i in range(critic_updates):
discriminator_optimizer.zero_grad()
# train discriminator on real and fake
if used_loss_function== 'wgan_gp_loss':
gp_lambda = 10
alpha = random.random()
interpolates = alpha * sharped_images + (1 - alpha) * deblurred_img
interpolates_discriminator_out = discriminator(interpolates)
kwargs = {'gp_lambda': gp_lambda,
'interpolates': interpolates,
'interpolates_discriminator_out': interpolates_discriminator_out,
'sharp_discriminator_out': sharp_discriminator_out,
'deblurred_discriminator_out': deblurred_discriminator_out
}
wgan_loss_d, gp_d = wgan_gp_loss('D', **kwargs)
discriminator_loss_per_update = wgan_loss_d + gp_d
discriminator_loss_per_update.backward(retain_graph=True)
discriminator_optimizer.step()
discriminator_loss += discriminator_loss_per_update.item()
But when I run this code, I receive the following error message:但是当我运行此代码时,我收到以下错误消息:
RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.cuda.FloatTensor [1, 512, 4, 4]] is at version 2;
RuntimeError:梯度计算所需的变量之一已被就地操作修改:[torch.cuda.FloatTensor [1, 512, 4, 4]] 为版本 2; expected version 1 instead.
而是预期的版本 1。 Hint: the backtrace further above shows the operation that failed to compute its gradient.
提示:上面的回溯显示了未能计算其梯度的操作。 The variable in question was changed in there or anywhere later.
有问题的变量在那里或以后的任何地方都被改变了。 Good luck!
祝你好运!
RuntimeError Traceback (most recent call last) in () 62 # discriminator_loss_per_update = gan_loss_d 63 —> 64 discriminator_loss_per_update.backward(retain_graph=True) 65 discriminator_optimizer.step() 66 discriminator_loss += discriminator_loss_per_update.item()
RuntimeError Traceback (最近一次调用最后一次) in () 62 # discriminator_loss_per_update = gan_loss_d 63 —> 64 discriminator_loss_per_update.backward(retain_graph=True) 65 discriminator_optimizer.step() 66 discriminator_loss += discriminator_loss_per_update.item()
1 frames /usr/local/lib/python3.7/dist-packages/torch/tensor.py in backward(self, gradient, retain_graph, create_graph, inputs) 243 create_graph=create_graph, 244 inputs=inputs) → 245 torch.autograd.backward(self, gradient, retain_graph, create_graph, inputs=inputs) 246 247 def register_hook(self, hook):
1 帧 /usr/local/lib/python3.7/dist-packages/torch/tensor.py 向后(自我,渐变,retain_graph,create_graph,输入)243 create_graph=create_graph,244 个输入=输入)→ 245 torch.autograd .backward(self,gradient,retain_graph,create_graph,inputs=inputs)246 247 def register_hook(self,hook):
/usr/local/lib/python3.7/dist-packages/torch/autograd/init.py in backward(tensors, grad_tensors, retain_graph, create_graph, grad_variables, inputs) 145 Variable.execution_engine.run_backward( 146 tensors, grad_tensors, retain_graph, create_graph, inputs, → 147 allow_unreachable=True, accumulate_grad=True) # allow_unreachable flag 148 149
/usr/local/lib/python3.7/dist-packages/torch/autograd/init.py 在后向(张量,grad_tensors,retain_graph,create_graph,grad_variables,inputs)145 Variable.execution_engine.run_backward(146张量,grad_tensors,retain_graph , create_graph, 输入, → 147 allow_unreachable=True, accumulate_grad=True) #allow_unreachable flag 148 149
Unfortunately, I can't really trace the in-place operation that would cause this error.不幸的是,我无法真正追踪会导致此错误的就地操作。 Does anybody maybe has an idea or advice for me?
有人可能对我有想法或建议吗? I would appreciate any input:slight_smile:
我将不胜感激任何输入:slight_smile:
try changing the last line to:尝试将最后一行更改为:
discriminator_loss = discriminator_loss + discriminator_loss_per_update.item()
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.