[英]Pytorch: RuntimeError: expected dtype Float but got dtype Long
I encounter this weird error when building a simple NN in Pytorch. I dont understand this error and why this consern Long and Float datatype in backward function. Anyone encounter this before?我在 Pytorch 中构建一个简单的 NN 时遇到了这个奇怪的错误。我不明白这个错误以及为什么这个 consern Long 和 Float 数据类型向后 function。以前有人遇到过这个吗? Thanks for any help.
谢谢你的帮助。
Traceback (most recent call last):
File "test.py", line 30, in <module>
loss.backward()
File "/home/liuyun/anaconda3/envs/torch/lib/python3.7/site-packages/torch/tensor.py", line 198, in backward
torch.autograd.backward(self, gradient, retain_graph, create_graph)
File "/home/liuyun/anaconda3/envs/torch/lib/python3.7/site-packages/torch/autograd/__init__.py", line 100, in backward
allow_unreachable=True) # allow_unreachable flag
RuntimeError: expected dtype Float but got dtype Long (validate_dtype at /opt/conda/conda-bld/pytorch_1587428398394/work/aten/src/ATen/native/TensorIterator.cpp:143)
frame #0: c10::Error::Error(c10::SourceLocation, std::string const&) + 0x4e (0x7f5856661b5e in /home/liuyun/anaconda3/envs/torch/lib/python3.7/site-packages/torch/lib/libc10.so)
frame #1: at::TensorIterator::compute_types() + 0xce3 (0x7f587e3dc793 in /home/liuyun/anaconda3/envs/torch/lib/python3.7/site
-packages/torch/lib/libtorch_cpu.so)
frame #2: at::TensorIterator::build() + 0x44 (0x7f587e3df174 in /home/liuyun/anaconda3/envs/torch/lib/python3.7/site-packages
/torch/lib/libtorch_cpu.so)
frame #3: at::native::smooth_l1_loss_backward_out(at::Tensor&, at::Tensor const&, at::Tensor const&, at::Tensor const&, long)
+ 0x193 (0x7f587e22cf73 in /home/liuyun/anaconda3/envs/torch/lib/python3.7/site-packages/torch/lib/libtorch_cpu.so)
frame #4: <unknown function> + 0xe080b7 (0x7f58576960b7 in /home/liuyun/anaconda3/envs/torch/lib/python3.7/site-packages/torc
h/lib/libtorch_cuda.so)
frame #5: at::native::smooth_l1_loss_backward(at::Tensor const&, at::Tensor const&, at::Tensor const&, long) + 0x16e (0x7f587
e23569e in /home/liuyun/anaconda3/envs/torch/lib/python3.7/site-packages/torch/lib/libtorch_cpu.so)
frame #6: <unknown function> + 0xed98af (0x7f587e71c8af in /home/liuyun/anaconda3/envs/torch/lib/python3.7/site-packages/torc
h/lib/libtorch_cpu.so)
frame #7: <unknown function> + 0xe22286 (0x7f587e665286 in /home/liuyun/anaconda3/envs/torch/lib/python3.7/site-packages/torc
h/lib/libtorch_cpu.so)
Here is the source code:这是源代码:
import torch
import torch.nn as nn
import numpy as np
import torchvision
from torchvision import models
from UTKLoss import MultiLoss
from ipdb import set_trace
# out features [13, 2, 5]
model_ft = models.resnet18(pretrained=True)
num_ftrs = model_ft.fc.in_features
model_ft.fc = nn.Linear(num_ftrs, 20)
model_ft.cuda()
criterion = MultiLoss()
optimizer = torch.optim.Adam(model_ft.parameters(), lr = 1e-3)
image = torch.randn((1, 3, 128, 128)).cuda()
age = torch.randint(110, (1,)).cuda()
gender = torch.randint(2, (1,)).cuda()
race = torch.randint(5, (1,)).cuda()
optimizer.zero_grad()
output = model_ft(image)
age_loss, gender_loss, race_loss = criterion(output, age, gender, race)
loss = age_loss + gender_loss + race_loss
loss.backward()
optimizer.step()
Here is what I define my loss function这是我定义的损失 function
import torch
import torch.nn as nn
import torch.nn.functional as F
class MultiLoss(nn.Module):
def __init__(self):
super().__init__()
def forward(self, output, age, gender, race):
age_pred = output[:, :13]
age_pred = torch.sum(age_pred, 1)
gender_pred = output[:, 13: 15]
race_pred = output[:, 15:]
age_loss = F.smooth_l1_loss(age_pred.view(-1, 1), age.cuda())
gender_loss = F.cross_entropy(gender_pred, torch.flatten(gender).cuda(), reduction='sum')
race_loss = F.cross_entropy(race_pred, torch.flatten(race).cuda(), reduction='sum')
return age_loss, gender_loss, race_loss
Change the criterion
call to:将
criterion
调用更改为:
age_loss, gender_loss, race_loss = criterion(output, age.float(), gender, race)
If you look at your error we can trace it to:如果您查看您的错误,我们可以将其追溯到:
frame #3: at::native::smooth_l1_loss_backward_out
In the MultiLoss Class, the smooth_l1_loss
works with age
.在 MultiLoss Class 中,
smooth_l1_loss
适用于age
。 So I changed it's type to float (as the expected dtype is Float) while passing it to the criterion
.所以我将它的类型更改为 float (因为预期的 dtype 是 Float),同时将它传递给
criterion
。 You can check that age is torch.int64
(ie torch.long
) by printing age.dtype
您可以通过打印
age.dtype
检查年龄是否为torch.int64
(即torch.long
)
I am not getting the error after doing this.执行此操作后我没有收到错误消息。 Hope it helps.
希望能帮助到你。
Check the data type of "output", "age", "gender", "race"检查“output”、“age”、“gender”、“race”的数据类型
there may be a difference like可能会有不同,比如
"torch.float32"
"torch.float64"
Set them in the same type.将它们设置为相同的类型。 it will fix the error
它会修复错误
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.