Predict using a single image tensorflow , keras

Question

I'm trying to build and train a model to predict American Sign Language (using Sign Language MNIST dataset). So far I have managed to build the model and used the build model to predict train dataset. Accuracy is also above 70 % in train images. Now I want to predict using a single image using the trained model. The thing is the predicted result (class name) is wrong. I followed this kernal. I want to predict the sign for any given image.

here is the code

train = pd.read_csv('../asl_data_train/sign-language-mnist/sign-mnist-train.csv')
test = pd.read_csv('../asl_data_train/sign-language-mnist/sign-mnist-test.csv')

train.head()

train.shape

labels = train['label'].values

unique_val = np.array(labels)
np.unique(unique_val)

plt.figure(figsize = (18,8))
sns.countplot(x =labels)

train.drop('label', axis = 1, inplace = True)

images = train.values
images = np.array([np.reshape(i, (28, 28)) for i in images])
images = np.array([i.flatten() for i in images])


label_binrizer = LabelBinarizer()
labels = label_binrizer.fit_transform(labels)

plt.imshow(images[0].reshape(28,28))


x_train, x_test, y_train, y_test = train_test_split(images, labels, test_size = 0.3, random_state = 101)


batch_size = 128
num_classes = 24
epochs = 50

x_train = x_train / 255
x_test = x_test / 255

x_train = x_train.reshape(x_train.shape[0], 28, 28, 1)
x_test = x_test.reshape(x_test.shape[0], 28, 28, 1)
plt.imshow(x_train[0].reshape(28,28))

code for build model

model = Sequential()
model.add(Conv2D(64, kernel_size=(3,3), activation = 'relu', input_shape=(28,28,1) ))
model.add(MaxPooling2D(pool_size = (2, 2)))

model.add(Conv2D(64, kernel_size = (3, 3), activation = 'relu'))
model.add(MaxPooling2D(pool_size = (2, 2)))

model.add(Conv2D(64, kernel_size = (3, 3), activation = 'relu'))
model.add(MaxPooling2D(pool_size = (2, 2)))

model.add(Flatten())
model.add(Dense(128, activation = 'relu'))
model.add(Dropout(0.20))

model.add(Dense(num_classes, activation = 'softmax'))

model.compile(loss = keras.losses.categorical_crossentropy, optimizer=keras.optimizers.Adam(),
              metrics=['accuracy'])

history = model.fit(x_train, y_train, validation_data = (x_test, y_test), epochs=epochs, batch_size=batch_size)
model.save("testmodel.h5")

prediction for test images

plt.plot(history.history['acc'])
plt.plot(history.history['val_acc'])
plt.title("Accuracy")
plt.xlabel('epoch')
plt.ylabel('accuracy')
plt.legend(['train','test'])
plt.show()

test_labels = test['label']

test.drop('label', axis = 1, inplace = True)

test_images = test.values
test_images = np.array([np.reshape(i, (28, 28)) for i in test_images])
test_images = np.array([i.flatten() for i in test_images])

test_labels = label_binrizer.fit_transform(test_labels)

test_images = test_images.reshape(test_images.shape[0], 28, 28, 1)

test_images.shape

y_pred = model.predict(test_images)

accuracy_score(test_labels, y_pred.round())

Here I get the accuracy score around 0.8...

This is how I tried to use a single image to predict the sign

 model = load_model("testmodel.h5")

test_image = image.load_img('a.jpg',color_mode="grayscale",target_size=(28,28,1))
print(test_image.format)
print(test_image.mode)
print(test_image.size)

test_image = image.img_to_array(test_image)
test_image = test_image / 255
test_image  = test_image.reshape((-1,) + test_image.shape)

print(test_image.dtype)
print(test_image.shape)

y_pred = model.predict_classes(test_image)
print(y_pred)
classname = y_pred[0]
print("Class: ",classname)

In here I get the classname but it changes for example for letter "A" (a.jpg) I get class 6. What am i doing wrong here.. please point me in the right direction.

Answer 1

Is the image "a.jpg" part of the same data set?

If the answer is no -> you should keep in mind that an NN can only predict images that have similar characteristics to the training images. If the NN is trained with a wide type of images, it could predict a wide range of images, but if the NN is trained with a very static data set (white background, same size, hand centered, etc.) it would fail if the Input image is very different.

If the answer is yes -> if you obtained 80% accuracy, the image may be misclassified. If you used a set of test data to validate your NN, you must obtain the same results by using them as a group or by passing them one by one.