Keras Multi-Class Image Segmentation - number of classes?

Question

Apologies if this is a stupid question but I have a dataset with two classes I wish to attempt to classify using a U-Net.

When creating the label matrices, do I need to explicitly define the null / base class (everything which isn't a class) or will Keras calculate this automatically?

For example, if I have a set of images where I'd like to classify the regions where there is a dog or where this is a cat , do I need to create a third label matrix which labels everything which is not a dog or cat (and thus, have three classes)?

Furthermore, the null class dominates the images I'm wishing to segment; if I were to use a class_weight, it seems to only accept a dictionary as input whereas I swear before I good specify a list and that would suffice.

If I treat my problem as a two-class problem, I'm assuming I need to specify the weight of the null class too, ie class_weight = [nullweight, dogweight, catweight] .

Thank you

edit: Attached example

Is this above image a two class or three class problem?

Answer 1

You must specify the other class since the network needs to differentiate between the dog, the cat and the background.

As for the class_weights parameter, the discussion is a little bit more complicated, you cannot assign like you would do in a simple classification problem.

Indeed, in many problems the background constitutes a big part of the image so you need to be careful when approaching such an imbalanced problem.

You need to inspect the parameter sample_weights , not class_weights , you can have a look at these threads:

1. https://datascience.stackexchange.com/questions/31129/sample-importance-training-weights-in-keras
2. https://github.com/keras-team/keras/issues/3653
3. Weighting samples in multiclass image segmentation using keras image-segmentation-using-keras