Keras: multiclass classification Dense layer

Question

I've seen that we can use Dense(num_classes, ...) as the output layer, but I've also seen Dense((num_classes-1), ...) especially when talking about binary classification. Which do you use and why?

Answer 1

Here is my 2-cent,

I use Dense(num_classes) because I can compute softmax on the output of this layer.

For a binary classification, we typically use cross-entropy as a loss function. Thus, we only need to compute p(x = 1) and p(x = 0) can be computed as 1 - p(x=1).

For a multiclass classification, the only advantage to reduce the size of output by 1 is to reduce the number parameters because we can compute p(x = k) = 1 - sum_{i=1}^K p(x=i), given that K is the number classes. So if you plan not to use softmax but a different function, maybe you can have the output layer to have (num_classes - 1) units.