[英]PyTorch nn.CrossEntropyLoss runtime dimension out of range error
I'm currently implementing the continuous bag-of-words (CBOW) model using PyTorch.我目前正在使用 PyTorch 实现连续词袋 (CBOW) model。 I'm facing some problems when implementing the cross entropy loss, though.
不过,在实现交叉熵损失时,我遇到了一些问题。 Here's the portion of code that's causing the problem:
这是导致问题的代码部分:
for idx, sample in enumerate(self.train_data):
x = torch.tensor(sample[0], dtype=torch.long)
y = np.zeros(shape=(self.vocab_size)) # self.vocab_size = 85,000
y[int(sample[1])] = np.float64(1)
y = torch.tensor(y, dtype=torch.long)
if torch.cuda.is_available():
x = x.cuda()
y = y.cuda()
optimizer.zero_grad()
output = self.model(x) # output's shape is the same as self.vocab_size
loss = criterion(output, y)
loss.backward()
optimizer.step()
To briefly explain my code, the model
that I've implemented basically outputs the averaged embedding values of a context array and performs a linear projection to project them into a shape that's identical to the size of the vocabulary.为了简要解释我的代码,我实现的
model
基本上输出上下文数组的平均嵌入值,并执行线性投影以将它们投影成与词汇表大小相同的形状。 Then we run this array through a softmax function.然后我们通过 softmax function 运行这个数组。
The contents of self.train_data
are basically (context, target_word)
pairs. self.train_data
的内容基本上是(context, target_word)
对。 y
is a one-hot encoded array of the token. y
是令牌的 one-hot 编码数组。
I'm aware that the second input to nn.CrossEntropyLoss
is C = # of classes
, but I'm not sure where my code went wrong.我知道 nn.CrossEntropyLoss 的第二个输入是
nn.CrossEntropyLoss
C = # of classes
,但我不确定我的代码哪里出错了。 The vocabulary size is 85,000 and so aren't the number of class 85,000?词汇量是 85,000,所以 class 的数量不是 85,000 吗?
If I change the input to如果我将输入更改为
loss = criterion(output, 85000)
I get the same error:我犯了同样的错误:
*** RuntimeError: Dimension out of range (expected to be in range of [-1, 0], but got 1)
What am I doing wrong, and how should I understand the input to PyTorch's cross entropy loss?我做错了什么,我应该如何理解 PyTorch 的交叉熵损失的输入?
Thanks.谢谢。
I'm aware that the second input to nn.CrossEntropyLoss is C = # of classes, but I'm not sure where my code went wrong.
我知道 nn.CrossEntropyLoss 的第二个输入是 C = # of classes,但我不确定我的代码哪里出错了。 The vocabulary size is 85,000 and so aren't the number of class 85,000?
词汇量是 85,000,所以 class 的数量不是 85,000 吗?
The number of classes (nc) may be the 85000, but you also have the batch size:类的数量(nc)可能是 85000,但你也有批量大小:
target = torch.randint (nc, (bs,))
The target represents the true value, while output is what you get from the model for the particular input x in your case output = self.model(x)
.目标代表真实值,而 output 是您从 model 获得的特定输入 x 在您的情况下
output = self.model(x)
。
In here在这里
loss = criterion(output, target)
You can say the output is what you currently get from the model, and the target is what you should get when you finalize your training.您可以说 output 是您目前从 model 获得的,目标是您在完成训练时应该获得的。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.