I want to change the specific prediction of my CNN-model to a probability

Question

I trained a model to categorize pictures in two different types. Everything is working quite good, but my Model can only do a specific prediction (1 or 0 in my case), but I am interested to have a prediction which is more like a probability (For example 90% 1 and 10% 0). Where is the part of my code which I should change now? Is it something with the sigmoid function in the end which decides if its 1 or 0? Help would be nice. Thanks in advance.

import numpy as np
from keras.callbacks import TensorBoard
from keras import regularizers
from keras.models import Sequential
from keras.layers import Activation, Dropout, Flatten, Dense, Conv2D, MaxPooling2D
from keras.optimizers import Adam
from keras.metrics import categorical_crossentropy
from keras.preprocessing.image import ImageDataGenerator, array_to_img, img_to_array, load_img
from keras.layers.normalization import BatchNormalization

from utils import DataGenerator, PATH

train_path = 'Dataset/train'
valid_path = 'Dataset/valid'
test_path = 'Dataset/test'

model = Sequential()
model.add(Conv2D(16, (3, 3), input_shape=(640, 640, 1), padding='same', activation='relu',
                 kernel_regularizer=regularizers.l2(1e-4),
                 bias_regularizer=regularizers.l2(1e-4)))
model.add(MaxPooling2D(pool_size=(4, 4)))

model.add(Conv2D(32, (3, 3), activation='relu',
                 kernel_regularizer=regularizers.l2(1e-4),
                 bias_regularizer=regularizers.l2(1e-4)))
model.add(MaxPooling2D(pool_size=(5, 5)))

model.add(Conv2D(64, (3, 3), activation='relu',
                 kernel_regularizer=regularizers.l2(1e-4),
                 bias_regularizer=regularizers.l2(1e-4)))
model.add(MaxPooling2D(pool_size=(6, 6)))

model.add(Flatten())
model.add(Dense(64, activation='relu',
                kernel_regularizer=regularizers.l2(1e-4),
                bias_regularizer=regularizers.l2(1e-4)))
model.add(Dropout(0.3))
model.add(Dense(1, activation='sigmoid',
                kernel_regularizer=regularizers.l2(1e-4),
                bias_regularizer=regularizers.l2(1e-4)))
print(model.summary())

model.compile(loss='binary_crossentropy', optimizer=Adam(lr=1e-3), metrics=['accuracy'])

epochs = 50
batch_size = 16

datagen = DataGenerator()
datagen.load_data()

model.fit_generator(datagen.flow(batch_size=batch_size), epochs=epochs, validation_data=datagen.get_validation_data(),
                    callbacks=[TensorBoard(log_dir=PATH+'/tensorboard')])

#model.save_weights('first_try.h5')
model.save('second_try')

If I try to get a picture in my model like this:

path = 'train/clean/picturenumber2'  
def prepare(filepath):   
  IMG_SIZE = 640    
  img_array = cv2.imread(filepath, cv2.IMREAD_GRAYSCALE)    
  new_array = cv2.resize(img_array, (IMG_SIZE, IMG_SIZE))   
  return new_array.reshape(-1, IMG_SIZE, IMG_SIZE, 1) 

model = tf.keras.models.load_model('second_try') 
prediction = model.predict(prepare(path))
print(prediction)

I just get an output like this: [[1.]] Also if I put in a list with multiple pictures. The prediction itself seems to be working.

Answer 1

short answer : change sigmoid activation function in the last layer to softmax

why ?

because sigmoid output range is 0.0 to 1.0, so to make a meaningful interpretation of this output, you choose an appropriate threshold above which represents the positive class and anything below as negative class.(for a binary classification problem)

even softmax has the same output range but the difference being its outputs are normalized class probabilities more on that here , so if your model outputs 0.99 on any given input, then it can be interpreted as the model is 99.0% confident that it is a positive class and 0.1% confident that it belongs to a negative class.

update : as @amin suggested, if you need normalized probabilities you should do couple more changes for it to work.

1. modify your data generator to output 2 classes/labels instead of one.

2. change last Dense layer from 1 node to 2 nodes.