Getting more than 100% accuracy for model.evaluate() for CNN, when I run the model multiple times. How do I separate each model.fit() instance?

Question

I am currently trying to run a CNN model that looks at images of cats and dogs to try to classify them (an old kaggle question). However, I want to see what the effects of running different numbers of epochs is on model accuracy.

Unfortunately, when I run model.fit() multiple times, alongside a model.evaluate(), the accuracy scores seems to be adding on top of one another from the previous model run. For example the INCORRECT output I get is:

100 Epochs Test Accuracy: 67.7%
200 Epochs Test Accuracy: 98.5%
300 Epochs Test Accuracy: 259.0%

My CNN is build as such:

input_layer = Input(shape=(img_shape, img_shape, 3))

convolution_layer_1 = Conv2D(32, kernel_size=(5,5), activation = 'relu')(input_layer)
max_pool_1 = MaxPooling2D(pool_size=(2,2), strides=2)(convolution_layer_1)

convolution_layer_2 = Conv2D(64, kernel_size=(5,5), activation = 'relu')(max_pool_1)
max_pool_2 = MaxPooling2D(pool_size=(2,2),strides=2)(convolution_layer_2)

dense_layer_1 = Dense(32, activation='relu')(max_pool_2)
flatten_layer_1 = Flatten()(dense_layer_1)
dropout_1 = Dropout(0.4)(flatten_layer_1)

output_layer = Dense(1, activation='softmax')(dropout_1)

model = Model(inputs=input_layer, outputs=output_layer)
model.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy'])
model.summary()

Then I ran the following series of model.fit() and model.evaluate() commands, with only the epochs value (and print statements) being changed each time.

model.fit_generator(
    training_set_data, 
    epochs = 100, 
    validation_data = test_set_data, 
)

score_100 = model.evaluate(test_set_data)
print('\nTest accuracy for 100 epochs: %0.4f%%' % (score_100[0] * 100))

# 100 Epochs Test Accuracy: 67.7%

-------------------------------------------

model.fit_generator(
    training_set_data, 
    epochs = 200, 
    validation_data = test_set_data, 
)

score_200 = model.evaluate(test_set_data)
print('\nTest accuracy for 200 epochs: %0.4f%%' % (score_200[0] * 100))

#200 Epochs Test Accuracy: 98.5%

-------------------------------------------

model.fit_generator(
    training_set_data, 
    epochs = 300, 
    validation_data = test_set_data, 
)

score_300 = model.evaluate(test_set_data)
print('\nTest accuracy for 300 epochs: %0.4f%%' % (score_300[0] * 100))

#300 Epochs Test Accuracy: 259.0%

How do I run these three epochs iterations of the model.fit() separately? Such that the output of the model.evaluate() is printed correctly? For example:

100 Epochs Test Accuracy: 67.7%
200 Epochs Test Accuracy: 70.5%
300 Epochs Test Accuracy: 74.0%

I tried creating 3 separate compilations of the CNN to run each individually, as such:

model1 = Model(inputs=input_layer, outputs=output_layer)
model1.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy'])

model2 = Model(inputs=input_layer, outputs=output_layer)
model2.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy'])

then

model1.fit ....etc
model2.fit ....etc

However now the problem seems to be with the model itself? Because if I run model3 directly (for 300 epochs) without running model1 and model2, I still get an accuracy score of 200+%

What's going on!?

Answer 1

It's hard to tell what the problem is in your case but your code, with some minor modifications, trains a model to recognize wether an image is IS_NUMBER or not.

import tensorflow as tf
import tensorflow_datasets as tfds
from tensorflow.keras import layers
from tensorflow import keras


(ds_train, ds_test), ds_info = tfds.load(
    'mnist',
    split=['train', 'test'],
    shuffle_files=True,
    as_supervised=True,
    with_info=True,
)

inputs_spec, outputs_spec = ds_train.element_spec

input_layer = layers.Input(shape=inputs_spec.shape)
x = input_layer

x = layers.Conv2D(32, kernel_size=(3,3))(x)
x = layers.MaxPooling2D(pool_size=(2,2), strides=2)(x)

x = layers.Conv2D(64, kernel_size=(3,3))(x)
x = layers.MaxPooling2D(pool_size=(2,2),strides=2)(x)

x = layers.Flatten()(x)
x = layers.Dense(32)(x)
x = layers.Dropout(0.1)(x)

x = layers.Dense(1, activation='sigmoid')(x)

model = keras.Model(inputs=input_layer, outputs=x)
model.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy'])

IS_NUMBER = 3

def is_number(inp, tar):
  return inp, tf.cast(tf.equal(tar, IS_NUMBER), dtype=tf.int32)

ds_train = ds_train.map(is_number).repeat()

train = ds_train.padded_batch(50)

model.fit(
    train, 
    steps_per_epoch=200, 
    epochs=5,
    class_weight={0: 0.9, 1: 0.1}
)

I get similar results using:

# ...

x = layers.Dense(2, activation='softmax')(x)

model = keras.Model(inputs=input_layer, outputs=x)
model.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy'

IS_NUMBER = 3

def is_number(inp, tar):
  return inp, tf.cast(tf.equal(tar, IS_NUMBER), dtype=tf.int32)

# ...

model.fit(
    train, 
    steps_per_epoch=200, 
    epochs=5
)

Maybe this helps you figuring out the issue with your own code/data.

Answer 2

您指的是错误的参数 score_300[0] 将为您提供评估数据集的损失值，而 score_300[1] 将为您提供所有三种情况下的准确度值您所指的是错误值 score_100、score_200 和 score_300