How to handle false predictions in a neural network built and trained with Keras?

Question

I am very new to neural networks and I tried a typical first example with help of some Internet-Blogs: Image Classification of cats or dogs. After training the neural network below I tried to identify some random pictures of cats/dogs which I found on Google and which are neither in my training_set nor in my test_set… I found out, that sometimes the network gives a right prediction (recognizing a dog when showing a dog) and unfortunately sometimes a false prediction ie I showed a picture of a cat and the network predicts a 'dog'. How do I handle such mistakes?

Adding all wrong pictures to the training_set or test_set and do the whole training process again? Or is there any other option to tell the network that it has made a false prediction and should adapt its weights?

#Part 1 - Import
from keras.models import Sequential
from keras.layers import Conv2D
from keras.layers import MaxPooling2D
from keras.layers import Flatten
from keras.layers import Dense
#Part 2 – Build Network
classifier = Sequential()
classifier.add(Conv2D(32, (3, 3), input_shape = (64, 64, 3), activation = 'relu'))
classifier.add(MaxPooling2D(pool_size = (2, 2)))
classifier.add(Conv2D(32, (3, 3), activation = 'relu'))
classifier.add(MaxPooling2D(pool_size = (2, 2)))
classifier.add(Flatten())
classifier.add(Dense(units = 128, activation = 'relu'))
classifier.add(Dense(units = 1, activation = 'sigmoid'))
classifier.compile(optimizer = 'adam', loss = 'binary_crossentropy', metrics = ['accuracy'])
#Part 3 - Training
from keras.preprocessing.image import ImageDataGenerator
train_datagen = ImageDataGenerator(rescale = 1./255, shear_range = 0.2, zoom_range = 0.2, horizontal_flip = True)
test_datagen = ImageDataGenerator(rescale = 1./255)
training_set = train_datagen.flow_from_directory('C:/…/KNNDaten/training_set', target_size = (64, 64), batch_size = 32, class_mode = 'binary')
test_set = test_datagen.flow_from_directory('C:/…/KNNDaten/test_set', target_size = (64, 64), batch_size = 32, class_mode = 'binary')

classifier.fit_generator(training_set, steps_per_epoch = 8000, epochs = 25, validation_data = test_set, validation_steps = 2000)

#Part 4 – Saving Model and weights 
model_json = classifier.to_json()
with open("model1.json", "w") as json_file:
 json_file.write(model_json)
 classifier.save_weights("model1.h5")

# Part 5 - Making new predictions
import numpy as np
from keras.preprocessing import image
test_image = image.load_img('C:/… /KNNDaten/single_prediction/cat_or_dog_1.jpg', target_size = (64, 64))
test_image = image.img_to_array(test_image)
test_image = np.expand_dims(test_image, axis = 0)
result = classifier.predict(test_image)
training_set.class_indices
if result[0][0] == 1:
    prediction = 'dog'
else:
    prediction = 'cat'

print("Image contains: " + prediction);

At the moment my training process looks like: Results of my training process: accuracy, ...

Thank you very much for your help!

Answer 1

The usual process is to add the incorrectly predicted images to the training data set and retrain the network with random weights or using the weights obtained previously with the new images and the old ones.

When training a network you don't need to initiate with random weitghs, you could use the previous weights, this is sometimes called Transfer Learning. It is important if you try to do this to also include the original images used to train the model, or at least a part of it, if you don't want to overfit the model.

As Dascienz comments using data augmentation techniques can also be very useful to get a better generalization, for example adding the new images and variation of them: rotations, translation, symmetries and rescaling.