val_loss and val_accuracy do not change in siamese network

Question

I implemented the siamese network, but val_loss and val_accuracy do not change. Dataset is for faces. I made pairs for labels and images, and I can see pairs in the test, so I do not have problems reading pairs. Code for the Siamese Network:

inputs = layers.Input(input Shape)

x = layers.Conv2D(64,(3,3),padding = 'same')(inputs)
x =layers.BatchNormalization()(x)
x =layers.Activation('relu')(x)
x =layers.Dropout(0.8)(x)

x =layers.Conv2D(64,(3,3),padding = 'same')(x)
x =layers.BatchNormalization()(x)
x =layers.Activation('relu')(x)
x =layers.Dropout(0.8)(x)
x =layers.MaxPool2D(pool_size = (2,2),strides=(2,2))(x)

x =layers.Conv2D(128,(3,3),padding = 'same')(x)
x =layers.BatchNormalization()(x)
x =layers.Activation('relu')(x)
x =layers.Dropout(0.8)(x)

x =layers.Conv2D(128,(3,3),padding = 'same')(x)
x =layers.BatchNormalization()(x)
x =layers.Activation('relu')(x)
x =layers.Dropout(0.8)(x)
x =layers.MaxPool2D(pool_size = (2,2),strides=(2,2))(x)

x =layers.Conv2D(256,(3,3),padding = 'same')(x)
x =layers.BatchNormalization()(x)
x =layers.Activation('relu')(x)
x =layers.Dropout(0.8)(x)

x =layers.Conv2D(256,(3,3),padding = 'same')(x)
x =layers.BatchNormalization()(x)
x =layers.Activation('relu')(x)
x =layers.Dropout(0.8)(x)

x =layers.Conv2D(256,(3,3),padding = 'same')(x)
x =layers.BatchNormalization()(x)
x =layers.Activation('relu')(x)
x =layers.Dropout(0.8)(x)
x =layers.MaxPool2D(pool_size = (2,2),strides=(2,2))(x)

x =layers.Conv2D(256,(3,3),padding = 'same')(x)
x =layers.BatchNormalization()(x)
x =layers.Activation('relu')(x)
x =layers.Dropout(0.8)(x)
x =layers.MaxPool2D(pool_size = (2,2),strides=(2,2))(x)

x =layers.Dense(4096, activation ='relu')(x)
x =layers.Dropout(0.5)(x)
x =layers.Dense(4096, activation ='relu')(x)
x =layers.Dropout(0.5)(x)

pooledOutput = layers.GlobalAveragePooling2D()(x)
outputs = layers.Dense(embeddingDim)(pooledOutput)
# Build the model
model = Model(inputs, outputs)
# Return the model to the calling function

print (model.summary())
return model  

then I used adam and binary_crossentropy for compile section:

opt = Adam(lr = 1e-1)
model.compile(loss="binary_crossentropy", optimizer=opt,
          metrics=["accuracy"])

history = model.fit([pairTrain[:, 0], pairTrain[:, 1]],labelTrain,
 validation_data=([pairTest[:, 0],  pairTest[:, 1]],  labelTest),batch_size=Config.BATCH_SIZE,epochs=Config.EPOCHS)

And the model summary and the number of parameters:

 Layer (type)                Output Shape              Param #   
 =================================================================
 input_3 (InputLayer)        [(None, 96, 96, 3)]       0         
                                                             
 conv2d (Conv2D)             (None, 96, 96, 64)        1792      
                                                             
 batch_normalization (BatchNormalization)  (None, 96, 96, 64)       256       
                                                    
                                                             
 activation (Activation)     (None, 96, 96, 64)        0         
                                                             
 dropout (Dropout)           (None, 96, 96, 64)        0         
                                                             
 conv2d_1 (Conv2D)           (None, 96, 96, 64)        36928     
                                                             
 batch_normalization_1(BatchNormalization) (None, 96, 96, 64)       256       
                                                 
                                                             
 activation_1 (Activation)   (None, 96, 96, 64)        0         
                                                             
 dropout_1 (Dropout)         (None, 96, 96, 64)        0         
                                                             
 max_pooling2d (MaxPooling2D  (None, 48, 48, 64)       0                                                                        
                                                             
 conv2d_2 (Conv2D)           (None, 48, 48, 128)       73856     
                                                             
 batch_normalization_2 (BatchNormalization)  (None, 48, 48, 128)      512       
                                                 
                                                             
 activation_2 (Activation)   (None, 48, 48, 128)       0         
                                                             
 dropout_2 (Dropout)         (None, 48, 48, 128)       0         
                                                             
 conv2d_3 (Conv2D)           (None, 48, 48, 128)       147584    
                                                             
 batch_normalization_3 (BatchNormalization)  (None, 48, 48, 128)      512       
                                             
                                                             
 activation_3 (Activation)   (None, 48, 48, 128)       0         
                                                             
 dropout_3 (Dropout)         (None, 48, 48, 128)       0         
                                                             
 max_pooling2d_1 (MaxPooling2D)  (None, 24, 24, 128)      0         
                                                    
 conv2d_4 (Conv2D)           (None, 24, 24, 256)       295168    
                                                             
 batch_normalization_4 (BatchNormalization) (None, 24, 24, 256)      1024      
                                                                                                             
 activation_4 (Activation)   (None, 24, 24, 256)       0         
                                                             
 dropout_4 (Dropout)         (None, 24, 24, 256)       0         
                                                             
 conv2d_5 (Conv2D)           (None, 24, 24, 256)       590080    
                                                             
 batch_normalization_5 (BatchNormalization) (None, 24, 24, 256)      1024                                            
                                                             
 activation_5 (Activation)   (None, 24, 24, 256)       0         
                                                             
 dropout_5 (Dropout)         (None, 24, 24, 256)       0         
                                                             
 conv2d_6 (Conv2D)           (None, 24, 24, 256)       590080    
                                                             
 batch_normalization_6 (BatchNormalization) (None, 24, 24, 256)      1024      
                                                                                                           
 activation_6 (Activation)   (None, 24, 24, 256)       0         
                                                             
 dropout_6 (Dropout)         (None, 24, 24, 256)       0         
                                                             
 max_pooling2d_2 (MaxPooling2D)  (None, 12, 12, 256)      0         
                                                        
                                                             
 conv2d_7 (Conv2D)           (None, 12, 12, 256)       590080    
                                                             
 batch_normalization_7(BatchNormalization)  (None, 12, 12, 256)      1024      
                                                 
                                                             
 activation_7 (Activation)   (None, 12, 12, 256)       0         
                                                             
 dropout_7 (Dropout)         (None, 12, 12, 256)       0         
                                                             
 max_pooling2d_3 (MaxPooling2D)  (None, 6, 6, 256)        0         
                                                              
                                                             
 dense (Dense)               (None, 6, 6, 4096)        1052672   
                                                             
 dropout_8 (Dropout)         (None, 6, 6, 4096)        0         
                                                             
 dense_1 (Dense)             (None, 6, 6, 4096)        16781312  
                                                             
 dropout_9 (Dropout)         (None, 6, 6, 4096)        0         
                                                             
 global_average_pooling2d (GlobalAveragePooling2D)  (None, 4096)             0         
                                           
                                                             
 dense_2 (Dense)             (None, 48)                196656    
                                                             
 =================================================================
 Total params: 20,361,840
 Trainable params: 20,359,024
 Non-trainable params: 2,816
 ____________________________

The output looks like in below:

    Epoch 310/400
    1/1 [==============================] - 0s 191ms/step - loss: 2.5433e-04 - accuracy:         1.0000 - val_loss: 3.7977 - val_accuracy: 0.5000
    Epoch 311/400
    1/1 [==============================] - 0s 82ms/step - loss: 2.5718e-04 - accuracy:   1.0000 - val_loss: 3.7978 - val_accuracy: 0.5000
    Epoch 312/400
    1/1 [==============================] - 0s 81ms/step - loss: 2.5694e-04 - accuracy: 1.0000 - val_loss: 3.7978 - val_accuracy: 0.5000
    Epoch 313/400
    1/1 [==============================] - 0s 82ms/step - loss: 2.5570e-04 - accuracy: 1.0000 - val_loss: 3.7979 - val_accuracy: 0.5000
    Epoch 314/400
    1/1 [==============================] - 0s 81ms/step - loss: 2.5392e-04 - accuracy: 1.0000 - val_loss: 3.7979 - val_accuracy: 0.5000
    Epoch 315/400
    1/1 [==============================] - 0s 82ms/step - loss: 2.5371e-04 - accuracy: 1.0000 - val_loss: 3.7980 - val_accuracy: 0.5000
    Epoch 316/400
    1/1 [==============================] - 0s 81ms/step - loss: 2.5867e-04 - accuracy: 1.0000 - val_loss: 3.7980 - val_accuracy: 0.5000
    Epoch 317/400
    1/1 [==============================] - 0s 81ms/step - loss: 2.5400e-04 - accuracy: 1.0000 - val_loss: 3.7981 - val_accuracy: 0.5000
    Epoch 318/400
    1/1 [==============================] - 0s 83ms/step - loss: 2.5599e-04 - accuracy:    1.0000 - val_loss: 3.7981 - val_accuracy: 0.5000
    Epoch 319/400
    1/1 [==============================] - 0s 83ms/step - loss: 2.5272e-04 - accuracy: 1.0000 - val_loss: 3.7982 - val_accuracy: 0.5000

How can I fix the problem?

Answer 1

For the SNN, you'll need to implement contrastive_loss instead of binary CE, and even you are supposed to define same models with the same structure and weights and then define lambda layer which will take the output of these models and calculate similarity by a lambda layer which calculate euclidean distance

but your model is just one side, and just one inputs is defined in keras.Model() and it will also need preprocess data to group them how they feed into the model and the labels which define similarity or not.but in fitting model you are feeding model with two inputs.

and even in your model structure, you implement global avg pool after dense layers although model is not defined to classifying and so not needed any dense layers at all?? and even Adam as optimizer set to learning rate = 1E-1?!