In the paper introducing focal loss, they state that the loss function is formulated as such:
Where
I found an implementation of it on a Github page from another author who used it in their paper . I tried the function out on a segmentation problem dataset I have and it seems to work quite well.
Below is the implementation:
def binary_focal_loss(pred, truth, gamma=2., alpha=.25):
eps = 1e-8
pred = nn.Softmax(1)(pred)
truth = F.one_hot(truth, num_classes = pred.shape[1]).permute(0,3,1,2).contiguous()
pt_1 = torch.where(truth == 1, pred, torch.ones_like(pred))
pt_0 = torch.where(truth == 0, pred, torch.zeros_like(pred))
pt_1 = torch.clamp(pt_1, eps, 1. - eps)
pt_0 = torch.clamp(pt_0, eps, 1. - eps)
out1 = -torch.mean(alpha * torch.pow(1. - pt_1, gamma) * torch.log(pt_1))
out0 = -torch.mean((1 - alpha) * torch.pow(pt_0, gamma) * torch.log(1. - pt_0))
return out1 + out0
The part that I don't understand is the calculation of pt_0 and pt_1. I created a small example for myself to try and figure it out but it still confuses me a bit.
# one hot encoded prediction tensor
pred = torch.tensor([
[
[.2, .7, .8], # probability
[.3, .5, .7], # of
[.2, .6, .5] # background class
],
[
[.8, .3, .2], # probability
[.7, .5, .3], # of
[.8, .4, .5] # class 1
]
])
# one-hot encoded ground truth labels
truth = torch.tensor([
[1, 0, 0],
[1, 1, 0],
[1, 0, 0]
])
truth = F.one_hot(truth, num_classes = 2).permute(2,0,1).contiguous()
print(truth)
# gives me:
# tensor([
# [
# [0, 1, 1],
# [0, 0, 1],
# [0, 1, 1]
# ],
# [
# [1, 0, 0],
# [1, 1, 0],
# [1, 0, 0]
# ]
# ])
pt_0 = torch.where(truth == 0, pred, torch.zeros_like(pred))
pt_1 = torch.where(truth == 1, pred, torch.ones_like(pred))
print(pt_0)
# gives me:
# tensor([[
# [0.2000, 0.0000, 0.0000],
# [0.3000, 0.5000, 0.0000],
# [0.2000, 0.0000, 0.0000]
# ],
# [
# [0.0000, 0.3000, 0.2000],
# [0.0000, 0.0000, 0.3000],
# [0.0000, 0.4000, 0.5000]
# ]
# ])
print(pt_1)
# gives me:
# tensor([[
# [1.0000, 0.7000, 0.8000],
# [1.0000, 1.0000, 0.7000],
# [1.0000, 0.6000, 0.5000]
# ],
# [
# [0.8000, 1.0000, 1.0000],
# [0.7000, 0.5000, 1.0000],
# [0.8000, 1.0000, 1.0000]
# ]
# ])
What I don't understand is why in pt_0 we are placing zeros where the torch.where statement is false, and in pt_1 we place ones. From how I understood the paper, I would have thought that instead of placing zeros or ones, you would place 1-p.
Can anyone help explain this to me?
So the part you try to understand is a procedure people usually do when they want zero out the additional calculations that not needed.
Take another look at the formula of pt
:
The following code is does exactly this by separate the two condition:
# if y=1
pt_1 = torch.where(truth == 1, pred, torch.ones_like(pred))
# otherwise
pt_0 = torch.where(truth == 0, pred, torch.zeros_like(pred))
Where it set to zero in pt_0
and one in pt_1
will result zero in output thus have no effect for contribute loss value, ie:
# Because pow(0., gamma) == 0. and log(1.) == 0.
# out1 == 0. if pt_1 == 1.
out1 = -torch.mean(alpha * torch.pow(1. - pt_1, gamma) * torch.log(pt_1))
# out0 == 0. if pt_0 == 0.
out0 = -torch.mean((1 - alpha) * torch.pow(pt_0, gamma) * torch.log(1. - pt_0))
And the reason for pt_0
to using value of p
instead of 1-p
is the same reason as your last question, ie:
1 - (1 - p) == 1 - 1 + p == p
So it can later calculate the FL(pt)
by:
# -a * pow(1 - (1 - p), gamma )* log(1 - p) == -a * pow(p, gamma )* log(1 - p)
out0 = -torch.mean((1 - alpha) * torch.pow(pt_0, gamma) * torch.log(1. - pt_0))
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.