为什么 tensorflow 模型总是预测同一类？

Question

my TensorFlow model is always predicting the same class with a confidant of 100%.

First a short description of my setting: Task is to do image classification with 7 classes and reading images from the webcam. For training, validation, and testing of the model, I'm using TensorFlow with data generators.

model = tf.keras.models.Sequential()
model.add(tf.keras.layers.Conv2D(32,(3,3), activation='elu',input_shape=(image_heigth,image_width,3)))
model.add(tf.keras.layers.Conv2D(64,(3,3), activation='elu'))
model.add(tf.keras.layers.MaxPooling2D((2,2)))
model.add(tf.keras.layers.Conv2D(128,(3,3), activation='elu'))
model.add(tf.keras.layers.MaxPooling2D((2,2)))
model.add(tf.keras.layers.Conv2D(256,(3,3), activation='elu'))
model.add(tf.keras.layers.Conv2D(256,(3,3), activation='elu'))
model.add(tf.keras.layers.MaxPooling2D((2,2)))
model.add(tf.keras.layers.Conv2D(512,(3,3), activation='elu'))
model.add(tf.keras.layers.MaxPooling2D((2,2)))
model.add(tf.keras.layers.Dropout(0.3))
model.add(tf.keras.layers.Flatten())
model.add(tf.keras.layers.Dense(512, activation='elu'))
model.add(tf.keras.layers.Dense(7, activation='softmax'))

model.summary()

model.compile(loss='categorical_crossentropy', optimizer=tf.keras.optimizers.RMSprop(lr=1e-4), metrics=['acc'])

Just for information this is my model. Training, validation and testing is done with this code:

train_datagen = tf.keras.preprocessing.image.ImageDataGenerator(rescale=1./255, rotation_range=40, shear_range=0.2,zoom_range=0.2, horizontal_flip=True)
validation_datagen = tf.keras.preprocessing.image.ImageDataGenerator(rescale=1./255)
test_datagen = tf.keras.preprocessing.image.ImageDataGenerator(rescale=1./255)
train_generator = train_datagen.flow_from_directory(train_dir, target_size=(image_heigth,image_width),batch_size=batch_size, class_mode='categorical', shuffle=True)
validation_generator = validation_datagen.flow_from_directory(validation_dir, target_size=(image_heigth,image_width), batch_size=5, class_mode='categorical')

for data_batch, labels_batch in train_generator:
    print('Shape des Datenstapels:', data_batch.shape)
    print('Shape des Klassenbzeichnungsstabels:', labels_batch.shape)
    break

history = model.fit(train_generator,steps_per_epoch=steps_per_epoch, epochs=epochs, validation_data=validation_generator,validation_steps=10, callbacks=callback_list)

# Testing the Model
test_generator = test_datagen.flow_from_directory(test_dir, target_size=(image_heigth,image_width), batch_size=5, class_mode='categorical')
test_loss, test_acc = model.evaluate(test_generator, steps=5)
predictions = model.predict(test_generator)
image_batch, label_batch = next (test_generator) 

I'm reaching a correct classification rate up to 90%. My loss function is down to something around 0,3. Debugging the testing and watching the predictions get expected values like [0.08;0.06;0.56;0.04;0.10;0.09;0.07].

In the end, I'm saving my model as an h5 with the TensorFlow method.

In another python program, I'm loading this h5 and want to predict the webcam image. But now the output is always [1.0;0.0;0.0;0.0;0.0;0.0;0.0]. Here is my code how I'm doing this:

 import numpy as np
import cv2
import tensorflow as tf

model = tf.keras.models.load_model('/home/poppe/Dokumente/Models/Proto2.h5')

classes = ['One', 'two', 'three', 'four', 'five', 'six', 'seven']

model.summary()

cap = cv2.VideoCapture(0)

while(True):
    # Capture frame-by-frame
    ret, frame = cap.read()
    # Our operations on the frame come here
    frame = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)
    rezised = cv2.resize(frame, (150, 150))
    expandArrayImage = np.expand_dims(rezised, axis=0)


    prediction = model.predict(expandArrayImage)

    print (np.max(prediction))
    print(classes[np.argmax(prediction)])

    # Display the resulting frame
    cv2.imshow('frame',frame)
    if cv2.waitKey(1) & 0xFF == ord('q'):
        break

# When everything is done, release the capture
cap.release()
cv2.destroyAllWindows()

As you can see, I'm using OpenCV to read from a webcam.

To solve my problem I have tried the following: Reduce my model to a minimum --> no effect

Reduce my problem to a binary classification --> no effect (always predicting one of the two classes with 100%)

Load and predict a single image --> no effect

Load and predict a single image directly after the code for testing (don't have to save and load the model) --> no effect, the image was one of my test images...By testing this image was classified correctly, loading it as a single image and use the predict method I've got the same error as bevor.

So because of testing is running as expected, I don't think I've got a problem with my data or model. Is there anything wrong with transforming the webcam image from OpenCV to the TensorFlow model?

Do you have any other ideas I can try to fix my problem?

Many thanks at all! :)

Answer 1

cv2.VideoCapture().read() will return a numpy array with values ranging in (0,255) but your model expects them to be in range (0, 1)

you can pass an image within the expected range like:

rezised = cv2.resize(frame, (150, 150)) / 255
expandArrayImage = np.expand_dims(rezised, axis=0)
# rest of the code