pytorch BCELOSS vs binary_cross_entropy：权重的使用不同

Question

I observed that the weights in BCELoss do weight each sample in the batch, but the weights in binary_cross_entropy weight the loss depending on if 0 or 1 was the true label. That is very weird, since the docs of binary_cross_entropy docs of binary_cross_entropy have a redirection link to the docs of BCELoss .

This is a short example that only in binary_cross_entropy the loss is affected if 1 or 0 was the true label with corresponding the same predictions:

import torch
import torch.nn.functional as F
from torch.nn import BCELoss

predictions1 = torch.tensor([[0.2,0.8]],dtype=torch.float)
predictions2 = torch.tensor([[0.8,0.2]],dtype=torch.float)
true_val1 = torch.tensor([[0,1]],dtype=torch.float)
true_val2 = torch.tensor([[1,0]],dtype=torch.float)
weights = torch.tensor([[0.6,5]],dtype=torch.float)

bce_loss = BCELoss(weight=weights)

bce_loss(predictions1, true_val1)
>>> out: tensor(0.6248)

bce_loss(predictions2, true_val2)
>>> out: tensor(0.6248)

F.binary_cross_entropy(input=predictions1, target=true_val1, weight=weights)
>>> out: tensor(0.6248)

F.binary_cross_entropy_with_logits(input=predictions2, target=true_val2, weight=weights)
>>> out: tensor(2.1067)

Do I just not see how this makes sense? Why are there even two methods for the binary cross entropy loss in pytorch?

nothing. I wasn't looking into the source code of pytorch.

Answer 1

BCELoss class forward method is based on the F.binary_cross_entropy link . So it returns the same result because it is the same. In the most cases we initiate the class and pass the model outputs.

class BCELoss(_WeightedLoss):
      def __init__(self, weight: Optional[Tensor] = None, size_average=None, reduce=None, reduction: str = 'mean') -> None:
          super(BCELoss, self).__init__(weight, size_average, reduce, reduction)


      def forward(self, input: Tensor, target: Tensor) -> Tensor:
          return F.binary_cross_entropy(input, target, weight=self.weight, reduction=self.reduction)

However there might be use cases there we want to use the function instead.

nn.BCEWithLogitsLoss includes sigmoid activation, so you don't need to add it explicitly, as shown with just BCE:

loss = F.binary_cross_entropy(torch.sigmoid(input), target)