为什么在 CNN 的 output 层中使用 softmax 而不是 sigmoid 时得到截然不同的结果？

Question

I have a simple model which classifies images of triangles and circles.

Code:

    model = Sequential([
    Conv2D(16, 3, padding='same', activation='relu', input_shape=(150, 150 ,3)),
    MaxPooling2D(),
    Conv2D(32, 3, padding='same', activation='relu'),
    MaxPooling2D(),
    Conv2D(64, 3, padding='same', activation='relu'),
    MaxPooling2D(),
    Flatten(),
    Dense(512, activation='relu'),
    Dense(1,activation='sigmoid'),])
    model.compile(optimizer='adam',
                  loss='binary_crossentropy',
                  metrics=['accuracy'])

    model.fit(Xtr,ytr,epochs=3,batch_size=10)

The performance on the test set is:

But when I change the activation function in the output layer into softmax , ie the last layer turns into Dense(1,activation='softmax') , the model's performance becomes

I made different dataset splits, the results remained roughly the same (the model with softmax activation performed equally badly). What is the issue?

Answer 1

Using softmax, with your current configuration, is actually forcing it to choose always only one class. It might be the reason you get always recall equal to zero for one class and one for the other class in your experience using softmax.

First, you need to change the loss. The binary_crossentropy is not supposed to be used for softmax. If you change the loss to categorical cross-entropy and make the DENSE of size 2 for the last layer( since you want to choose between two classes using softmax), you should get almost the same performance; ie: change this part of the code;

Dense(1,activation='sigmoid'),])
model.compile(optimizer='adam',
              loss='binary_crossentropy',
              metrics=['accuracy'])

with this one:

Dense(2,activation='softmax'),])
model.compile(optimizer='adam',
              loss='categorical_crossentropy',
              metrics=['accuracy'])

Answer 2

model = Sequential([
Conv2D(16, 3, padding='same', activation='relu', input_shape=(150, 150 ,3)),
MaxPooling2D(),
Conv2D(32, 3, padding='same', activation='relu'),
MaxPooling2D(),
Conv2D(64, 3, padding='same', activation='relu'),
MaxPooling2D(),
Flatten(),
Dense(512, activation='relu'),
#Dense(1,activation='sigmoid'),]) #this means you have 1 node in last layer
                                  #that's why you have result of 1 class
Dense(2,activation='softmax'),])  #this is meaninful for 2 class. it will work on your code
model.compile(optimizer='adam',
              loss='categorical_crossentropy',
              metrics=['accuracy'])

model.fit(Xtr,ytr,epochs=3,batch_size=10)