Keras model always predicting 0

Question

so I have data like in the shape of (10000, 178, 178, 3) where I have 10000 samples and each have 3 different color channel(not the RGB one), where I have around 8k samples with label 0 and rest 2k as label 1 . Here's one of my sample data:

array([[[[1.79844797e-01, 1.73587397e-01, 1.73587397e-01, ...,
          4.84393053e-02, 5.15680127e-02, 5.46967126e-02],
         [1.76716089e-01, 1.79844797e-01, 1.82973504e-01, ...,
          5.15680127e-02, 5.31323589e-02, 5.15680127e-02],
         [1.81409150e-01, 1.86102197e-01, 1.81409150e-01, ...,
          5.15680127e-02, 5.31323589e-02, 5.15680127e-02]]],


       [[[2.51065755e+00, 2.53197193e+00, 2.53197193e+00, ...,
          1.88543844e+00, 1.89964795e+00, 1.90675282e+00],
         [2.51776242e+00, 2.52486706e+00, 2.53197193e+00, ...,
          1.89964795e+00, 1.90675282e+00, 1.90675282e+00],
         [2.53197193e+00, 2.51776242e+00, 2.52486706e+00, ...,
          1.91385746e+00, 1.90675282e+00, 1.90675282e+00]]],


       [[[7.13270283e+00, 7.11016369e+00, 7.13270283e+00, ...,
          4.85625362e+00, 4.90133190e+00, 4.94641018e+00],
         [7.08762503e+00, 7.08762503e+00, 7.08762503e+00, ...,
          4.92387104e+00, 4.96894932e+00, 4.96894932e+00],
         [7.08762503e+00, 7.08762503e+00, 7.06508589e+00, ...,
          4.99148846e+00, 4.96894932e+00, 4.96894932e+00]]],
      dtype=float32)

Now firstly I'm trying to normalize by color channel. As each color channel is completely different so I'm normalizing by color channel as follows, dara_array is my whole dataset:

def nan(index):
    data_array[:, :, :, index] = (data_array[:, :, :, index] - np.min(data_array[:, :, :, index]))/(np.max(data_array[:, :, :, index]) - np.min(data_array[:, :, : ,index]))
    

Splitting for training and testing:

rand_indices = np.random.permutation(len(data))
train_indices = rand_indices[0:7460]
valid_indices = rand_indices[7460:len(data)]

x_test = data_array[valid_indices, :]
y_test = EDR[[valid_indices]].astype('float')

x_train = data_array[train_indices, :]
y_train = EDR[[train_indices]].astype('float')

Then I'm using this Neural Network for training this dataset:

weight_decay = 1e-4
model = Sequential()
model.add(Conv2D(32, (20,20), padding='same', kernel_regularizer=regularizers.l2(weight_decay), input_shape=x_tr.shape[1:]))
model.add(LeakyReLU(alpha=0.01))
model.add(BatchNormalization())
model.add(Conv2D(32, (30,30), padding='same', kernel_regularizer=regularizers.l2(weight_decay)))
model.add(LeakyReLU(alpha=0.01))
model.add(BatchNormalization())
model.add(MaxPooling2D(pool_size=(2,2)))
model.add(Dropout(0.2))

model.add(Flatten())
model.add(Dense(1, activation='sigmoid'))

model.summary()

Then here I'm training it:

def lr_schedule(epoch):
    lrate = 0.001
    if epoch > 75:
        lrate = 0.0005
    elif epoch > 100:
        lrate = 0.0003        
    return lrate

batch_size = 128

opt_rms = tf.keras.optimizers.Adam()

model.compile(loss= 'binary_crossentropy', optimizer = opt_rms, metrics=['accuracy', tf.keras.metrics.Precision(), tf.keras.metrics.Recall()])

history = model.fit(x_train, y_train, batch_size, epochs=10, verbose=1,
                   callbacks=[LearningRateScheduler(lr_schedule)])

Here's the result of my all epoch:

Epoch 1/10
59/59 [==============================] - 420s 7s/step - loss: 6.7227 - accuracy: 0.7263 - precision_2: 0.2697 - recall_2: 0.2846 - lr: 0.0010
Epoch 2/10
59/59 [==============================] - 399s 7s/step - loss: 2.7919 - accuracy: 0.7440 - precision_2: 0.3027 - recall_2: 0.2991 - lr: 0.0010
Epoch 3/10
59/59 [==============================] - 399s 7s/step - loss: 2.9244 - accuracy: 0.7484 - precision_2: 0.3210 - recall_2: 0.3282 - lr: 0.0010
Epoch 4/10
59/59 [==============================] - 399s 7s/step - loss: 3.5013 - accuracy: 0.7509 - precision_2: 0.3246 - recall_2: 0.3261 - lr: 0.0010
Epoch 5/10
59/59 [==============================] - 398s 7s/step - loss: 3.1829 - accuracy: 0.7413 - precision_2: 0.3137 - recall_2: 0.3406 - lr: 0.0010
Epoch 6/10
59/59 [==============================] - 398s 7s/step - loss: 4.9515 - accuracy: 0.7592 - precision_2: 0.3307 - recall_2: 0.2999 - lr: 0.0010
Epoch 7/10
59/59 [==============================] - 398s 7s/step - loss: 2.3082 - accuracy: 0.7613 - precision_2: 0.3539 - recall_2: 0.3588 - lr: 0.0010
Epoch 8/10
59/59 [==============================] - 399s 7s/step - loss: 1.8624 - accuracy: 0.7520 - precision_2: 0.3273 - recall_2: 0.3282 - lr: 0.0010
Epoch 9/10
59/59 [==============================] - 398s 7s/step - loss: 2.7749 - accuracy: 0.7579 - precision_2: 0.3344 - recall_2: 0.3173 - lr: 0.0010
Epoch 10/10
59/59 [==============================] - 399s 7s/step - loss: 2.5800 - accuracy: 0.7513 - precision_2: 0.3288 - recall_2: 0.3362 - lr: 0.0010

Now when I'm printing classification report everything is coming 0 :

y_pred = model.predict(x_test)
y_pred_bool = np.argmax(y_pred, axis=1)

print(classification_report(y_test, y_pred_bool))

Output:

    precision    recall  f1-score   support

         0.0       0.82      1.00      0.90      2030
         1.0       0.00      0.00      0.00       453

    accuracy                           0.82      2483
   macro avg       0.41      0.50      0.45      2483
weighted avg       0.67      0.82      0.74      2483

Can someone tell me what I'm doing wrong or what I'm missing, Am I doing something wrong while normalizing the data or while training or is there something wrong with my model?

Here's one sample image from my data:

Answer 1

You are using an argmax on a single output, so it will only ever predict class 0. You can see this is the case by analysing your y_pred variable, or by switching your class labels and seeing that it will continue the trend.

To get around this, use a threshold instead of argmax or change your model to predict 1 output per class. There are ways to optimise this but for an initial test you can use a threshold of 0.5 or so like this:

T=0.5
y_pred = model.predict(x_test)
y_pred_bool = y_pred>=T

print(classification_report(y_test, y_pred_bool))