Opposite to expected behavior [PyTorch]

Question

I have a dataset that has severe class imbalance (bank-additional-full). I'm using it without making any changes except assigning values to categorical variable. It has about 89% classes of no(0) and 11% class of yes(1).

My model always predicts yes (lower count of one) and also changing learning rate is having no effect. It should predict the class that has larger count more often ie, no

I'm learning pytorch so let me know my mistake as i'm having trouble finding it.

class LogisticRegressionModel(nn.Module):
  def __init__(self, input_dim, output_dim):
      super(LogisticRegressionModel, self).__init__()
      self.linear = nn.Linear(input_dim, output_dim)
  def forward(self, x):

      out = F.softmax(self.linear(x),dim=1)
      return out

 input_dim = 1*20 
 output_dim = 2

 model = LogisticRegressionModel(input_dim, output_dim)
 criterion = nn.CrossEntropyLoss()
 learning_rate = 0.01
 optimizer = torch.optim.SGD(model.parameters(), lr=learning_rate)

 iter = 0
num_epochs= 10
train_losses, val_losses = [], []
for epoch in range(num_epochs):
  running_loss = 0
  for i, (x, labels) in enumerate(train_loader):

      x = Variable(x.view(-1, height*width))
      labels = Variable(labels)     
      optimizer.zero_grad()        
      outputs = model(x.float())       
      loss = criterion(outputs, labels)      
      loss.backward()

      optimizer.step()

      running_loss += loss.item()

      iter += 1

  else:
      val_loss = 0


      correct = 0
      total = 0
      with torch.no_grad():

          for x, labels in val_loader:

              x = Variable(x.view(-1,height*width))                             
              outputs = model(x.float())

              val_loss += criterion(outputs, labels)

              values, predicted = torch.max(outputs.data, 1)
              #print(values.data)    


              total += labels.size(0)

              correct += (predicted == labels).sum()                


      accuracy = 100 * correct / total

      train_losses.append(running_loss/len(train_loader))
      val_losses.append(val_loss/len(val_loader))

      print("Epoch: {}/{}.. ".format(epoch+1, num_epochs),
                "Training Loss: {:.3f}.. ".format(running_loss/len(train_loader)),
                "Validation Loss: {:.3f}.. ".format(val_loss/len(val_loader)),
                "Validation Accuracy: {:.3f}".format(accuracy))

      print("\n")

Results: Epoch: 1/10.. Training Loss: 1.201.. Validation Loss: 1.202.. Validation Accuracy: 11.000

loss and accuracy remain same and do not change

Answer 1

In the given code Weights are not being updated. If you run

list(model.parameters())[0].grad

You will get

tensor([[0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
        [0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.]])

or None. As gradients are zero weights are not updating.

If You change the F.softmax(out) to F.log_softmax(out) it fixes the problem. At the moment I do not have in depth knowledge of why it happens. But you can visit: https://discuss.pytorch.org/t/weights-never-update-while-training/8418/9

This link has some explanation about it

Edit:

Comment given by the @jodag explains the real problem.