Error making prediction with Keras model

Question

I'm trying to get a prediction on a sample using a pre-trained Keras model but am getting an error. I've detailed parts of the model-training script to show the data-preparation, matrix-shapes and model-specification;

Matrix Shapes & Data Preparation:

from __future__ import print_function
#import numpy as np
np.random.seed(1337)  # for reproducibility

from keras.datasets import mnist
from keras.models import Sequential
from keras.layers import Dense, Dropout, Activation, Flatten
from keras.layers import Convolution2D, MaxPooling2D
from keras.utils import np_utils
from keras import backend as K

batchsize = 128
nb_classes = 3
nb_epochs = 12

# input image dimensions
img_rows, img_cols = 28, 28
# number of convolutional filters to use
nb_filters = 32
# size of pooling area for max pooling
pool_size = (2, 2)
# convolution kernel size
kernel_size = (3, 3)

# the data, shuffled and split between train and test sets
#(X_train, y_train), (X_test, y_test) = mnist.load_data()

if K.image_dim_ordering() == 'th':
    X_train = X_train.reshape(X_train.shape[0], 1, img_rows, img_cols)
    X_test = X_test.reshape(X_test.shape[0], 1, img_rows, img_cols)
    input_shape = (1, img_rows, img_cols)
else:
    X_train = X_train.reshape(X_train.shape[0], img_rows, img_cols, 1)
    X_test = X_test.reshape(X_test.shape[0], img_rows, img_cols, 1)
    input_shape = (img_rows, img_cols, 1)

X_train = X_train.astype('float32')
X_test = X_test.astype('float32')
X_train /= 255
X_test /= 255
print('X_train shape:', X_train.shape)
print(X_train.shape[0], 'train samples')
print(X_test.shape[0], 'test samples')

# convert class vectors to binary class matrices
Y_train = np_utils.to_categorical(y_train, nb_classes)
Y_test = np_utils.to_categorical(y_test, nb_classes)

Model Specification:

model = Sequential()

model.add(Convolution2D(nb_filters, [kernel_size[0], kernel_size[1]],
                        padding='valid',
                        input_shape=input_shape,
                        name='conv2d_1'))
model.add(Activation('relu'))
model.add(Convolution2D(nb_filters, [kernel_size[0], kernel_size[1]], name='conv2d_2'))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=pool_size, name='maxpool2d'))
model.add(Dropout(0.25))

model.add(Flatten())
model.add(Dense(128, name='dense_1'))
model.add(Activation('relu'))
model.add(Dropout(0.5))
model.add(Dense(nb_classes, name='dense_2'))
model.add(Activation('softmax'))

model.compile(loss='categorical_crossentropy',
              optimizer='adadelta',
              metrics=['accuracy'])

In a entirely separate program the pre-trained model is reloaded, the input sample matrix is reshaped to match what the model expects and the the same normalization applied to the data. Like so;

Prediction Method:

from keras import backend as K
from keras.models import load_model

img_rows, img_cols = 28, 28

#Load the pre-trained classifier model
retrieved_model = load_model('classifier_cnn_saved_model_0.05_30min.hdf5')

#Function to callback
def get_prediction(sample):
    print('Received: ' + str(sample.shape))
    if K.image_dim_ordering() == 'th':
        sample = sample.reshape(sample.shape[0], 1, img_rows, img_cols)
    else:
        sample = sample.reshape(sample.shape[0], img_rows, img_cols, 1)

    print('Reshaped for backend: ' + K.image_dim_ordering() + ' ' + str(sample.shape))
    sample = sample.astype('float32')   
    sample /= 255 #normalize the sample data
    prediction = retrieved_model.predict(sample)
    print('pyAgent; ' + str(sample.shape) + ' prediction: ' + str(prediction))

Which gives this output when get_prediction is called;

Received: (1, 784) <====== Yep, as expected.
Reshaped for backend: tf (1, 28, 28, 1) <====== What the model expects, I think. Based on how it was specified at training time.

but then this error when the prediction is attempted;

Exception: ValueError: Tensor Tensor("activation_4/Softmax:0", shape=(?, 3), dtype=float32) is not an element of this graph. 

I'm stumped. Can anyone please point out what's wrong here and how to correct it? Many thanks.

NB All the training and the predictions are happening on the same Windows 10 machine using Python 3 with Keras 2.1.3 and Tensorflow 1.5.0

Answer 1

Considering this github issue yielded the answer. It is the case here that get_prediction() will be called by a different thread than the one which loaded the model. Making these changes cleared the error:

import tensorflow as tf #<======= add this
from keras import backend as K
from keras.models import load_model

img_rows, img_cols = 28, 28

#Load the pre-trained classifier model
retrieved_model = load_model('classifier_cnn_saved_model_0.05_30min.hdf5')
#https://www.tensorflow.org/api_docs/python/tf/Graph
graph = tf.get_default_graph() #<======= do this right after constructing or loading the model

#Function to callback
def get_prediction(sample):
    print('Received: ' + str(sample.shape))
    if K.image_dim_ordering() == 'th':
        sample = sample.reshape(sample.shape[0], 1, img_rows, img_cols)
    else:
        sample = sample.reshape(sample.shape[0], img_rows, img_cols, 1)

    print('Reshaped for backend: ' + K.image_dim_ordering() + ' ' + str(sample.shape))
    sample = sample.astype('float32')   
    sample /= 255 #normalize the sample data
    with graph.as_default(): #<======= with this, call predict
        prediction = retrieved_model.predict_classes(sample)
    print('pyAgent; ' + str(sample.shape) + ' prediction: ' + str(prediction))