移動到GPU時，張量類型不匹配

Question

嘗試將我的網絡和張量移動到GPU時，我收到以下錯誤。 我已經檢查過網絡參數是否已移動到GPU並檢查每個批次的張量並移動它們（如果它們尚未在GPU上）。 但我仍然得到這個問題，說張量類型不匹配 - 一個是torch.cuda.FloatTensor ，另一個是torch.FloatTensor ？ 有人能告訴我我做錯了什么嗎？ 謝謝。

我的代碼：

class Train():
  def __init__(self, network, training, address):
    self.network    = network
    self.address    = address
    self.batch_size = training['batch_size']
    self.iterations = training['iterations']
    self.samples    = training['samples']
    self.data       = training['data']
    self.lr         = training['lr']
    self.noisy_lr   = training['nlr']
    self.cuda       = training['cuda']
    self.save       = training['save']
    self.scale      = training['scale']
    self.limit      = training['limit']
    self.replace    = training['strategy']
    self.optimizer  = torch.optim.Adam(self.network.parameters(), lr=self.lr)

  def tensor_to_Variable(self, t):
    if next(self.network.parameters()).is_cuda and not t.is_cuda:
        t = t.cuda()

    return Variable(t)

  def train(self):
    if self.cuda:
        self.network.cuda()
    dh = DataHandler(self.data)
    loss_fn = torch.nn.MSELoss()
    losses    = []
    validate  = []
    val_size  = 100
    val_diff  = 1
    total_val = float(val_size * self.batch_size)
    hypos     = []
    labels    = []

    # training loop
    for i in range(self.iterations):
        x, y = dh.get_batch(self.batch_size)
        x = self.tensor_to_Variable(x)
        y = self.tensor_to_Variable(y)

        self.optimizer.zero_grad()
        hypo = self.network(x)
        loss = loss_fn(hypo, y)
        loss.backward()
        self.optimizer.step()


class Feedforward(nn.Module):
   def __init__(self, topology):
    super(Feedforward, self).__init__()
    self.input_dim     = topology['features']
    self.num_hidden    = topology['hidden_layers']
    self.hidden_dim    = topology['hidden_dim']
    self.output_dim    = topology['output_dim']
    self.input_layer   = nn.Linear(self.input_dim, self.hidden_dim)
    self.hidden_layer  = nn.Linear(self.hidden_dim, self.hidden_dim)
    self.output_layer  = nn.Linear(self.hidden_dim, self.output_dim)
    self.dropout_layer = nn.Dropout(p=0.2)


def forward(self, x):
    batch_size = x.size()[0]
    feat_size  = x.size()[1]
    input_size = batch_size * feat_size

    self.input_layer = nn.Linear(input_size, self.hidden_dim)
    hidden = self.input_layer(x.view(1, input_size)).clamp(min=0)

    for _ in range(self.num_hidden):
        hidden = self.dropout_layer(F.relu(self.hidden_layer(hidden)))

    output_size = batch_size * self.output_dim
    self.output_layer = nn.Linear(self.hidden_dim, output_size)
    return self.output_layer(hidden).view(output_size)

錯誤：

Traceback (most recent call last):
  File "/media/project/train.py", line 78, in train
    hypo = self.network(x)

 * (torch.cuda.FloatTensor mat1, torch.cuda.FloatTensor mat2)
 * (torch.cuda.sparse.FloatTensor mat1, torch.cuda.FloatTensor mat2)
 * (float beta, torch.cuda.FloatTensor mat1, torch.cuda.FloatTensor mat2)
 * (float alpha, torch.cuda.FloatTensor mat1, torch.cuda.FloatTensor mat2)
 * (float beta, torch.cuda.sparse.FloatTensor mat1, torch.cuda.FloatTensor mat2)
 * (float alpha, torch.cuda.sparse.FloatTensor mat1, torch.cuda.FloatTensor mat2)
 * (float beta, float alpha, torch.cuda.FloatTensor mat1, torch.cuda.FloatTensor mat2)
      didn't match because some of the arguments have invalid types: (int, int, torch.cuda.FloatTensor, torch.FloatTensor)
 * (float beta, float alpha, torch.cuda.sparse.FloatTensor mat1, torch.cuda.FloatTensor mat2)
      didn't match because some of the arguments have invalid types: (int, int, torch.cuda.FloatTensor, torch.FloatTensor)

堆棧跟蹤：

Traceback (most recent call last):
File "smpl.py", line 90, in <module>
main()
File "smpl.py", line 80, in main
trainer.train()
File "/media/mpl/temp/train.py", line 82, in train
hypo = self.network(x)
File "/usr/local/lib/python2.7/dist-packages/torch/nn/modules/module.py", line 206, in call
result = self.forward(input, **kwargs)
File "model/network.py", line 35, in forward
hidden = self.input_layer(x.view(1, input_size)).clamp(min=0)
File "/usr/local/lib/python2.7/dist-packages/torch/nn/modules/module.py", line 206, in call
result = self.forward(input, *kwargs)
File "/usr/local/lib/python2.7/dist-packages/torch/nn/modules/linear.py", line 54, in forward
return self.backend.Linear()(input, self.weight, self.bias)
File "/usr/local/lib/python2.7/dist-packages/torch/nn/_functions/linear.py", line 10, in forward
output.addmm(0, 1, input, weight.t())
TypeError: addmm_ received an invalid combination of arguments - got (int, int, torch.cuda.FloatTensor, torch.FloatTensor), but expected one of: (torch.cuda.FloatTensor mat1, torch.cuda.FloatTensor mat2)
(torch.cuda.sparse.FloatTensor mat1, torch.cuda.FloatTensor mat2) (float beta, torch.cuda.FloatTensor mat1, torch.cuda.FloatTensor mat2)
(float alpha, torch.cuda.FloatTensor mat1, torch.cuda.FloatTensor mat2) (float beta, torch.cuda.sparse.FloatTensor mat1, torch.cuda.FloatTensor mat2)
(float alpha, torch.cuda.sparse.FloatTensor mat1, torch.cuda.FloatTensor mat2) (float beta, float alpha, torch.cuda.FloatTensor mat1, torch.cuda.FloatTensor mat2)
didn't match because some of the arguments have invalid types: (int, int, torch.cuda.FloatTensor, torch.FloatTensor)
* (float beta, float alpha, torch.cuda.sparse.FloatTensor mat1, torch.cuda.FloatTensor mat2)
didn't match because some of the arguments have invalid types: (int, int, torch.cuda.FloatTensor, torch.FloatTensor

Answer 1

發生這種情況是因為您正在重新初始化 forward()函數中的self.input_layer 。

調用self.network.cuda()將所有模型參數移動到cuda中。 這意味着您在創建FeedForward對象時初始化的任何和所有圖層都將移動到cuda內存中。 但是當你在forward()函數中重新初始化 self.input_layer時，你在cpu而不是gpu中初始化該層的參數。 self.output_layer 。

Answer 2

首先，要使用GPU進行計算，您必須將數據類型准備到CUDA張量。

在這種情況下，可以簡單地完成如下操作。

dtype=torch.cuda.FloatTensor
x=torch.autograd.Variable(x.type(dtype))

您可以在tensor_to_Variable函數中根據此進行更改。

其次，要指定您希望您的“網絡”期望CUDA張量， network.cuda()將有所幫助。

最后，雖然這不是您問題的一部分，但在配置前饋網絡時無需指定批量大小。 為了闡明，

1）正面傳球：

def forward(self,x):
    x=self.input_layer(x)
    x=self.middle_layer(x)
    x=self.output_layer(x)
    return x

2）網絡初始化

def__init__(self,feature_size,hidden_size,output_size):
     self.input_layer=nn.Linear(feature_size,hidden_size)
     self.middle_layer=nn.Linear(hidden_size,hidden_size)
     self.output_layer=nn.Linear(hidden_size,output_size)

3）在打包到CUDA變量之前預處理數據

your_tensor.view(batch_size,feature_size)

希望這可以幫助！