深度学习分类中如何处理类之间的相关性？

Question

I am not sure about how to deal with the situation where some of the classes in a deep learning classification network are correlated to each other.

For example, when classifying images, if the input is a pic of a dog, and the output label is a cat, then the loss should be lower than if the output label is an airplane or a bridge. I was wondering how cs people deal with this before, and if there is any paper I can refer to. Thank you!

Answer 1

tl;dr:
You can't/shouldn't

Long version:
If, for your specific case, you can find a measurement to determine how close the classes are from each other, like this:

d[i,j] = f(L[i], L[j])

(where d[i,j] is the distance between the classes with labels L[i] and L[j] and f is a function of those 2 classes/labels) then you can incorporate this in your loss function, such that the loss will be greater if d is large and smaller if d is small.

However, in your example with cats/dogs/airplanes/bridges/... I can't believe there is a correct way to determine the distance between dog/cat or dog/bridge etc. If you really want to, you could of course manually decide that for example predicting a bridge is 3 times worse than predicting a dog if your image contains a cat:

d[cat,dog]=1
d[cat,bridge]=3

But I wouldn't recommend this at all.

When would I recommend this? If you have a number of classes and a correlation matrix that correctly quantifies the correlation between all classes.