different prediction after load a model in keras

Question

I have a Sequential Model built in Keras and after trained it give me good prediction but when i save and then load the model i don't obtain the same prediction on the same dataset. Why? Note that I checked the weight of the model and they are the same as well as the architecture of the model, checked with model.summary() and model.getWeights(). This is very strange in my opinion and I have no idea how to deal with this problem. I don't have any error but the prediction are different

1. I tried to use model.save() and load_model()

2. I tried to use model.save_weights() and after that re-built the model and then load the model

I have the same problem with both options.

def Classifier(input_shape, word_to_vec_map, word_to_index, emb_dim, num_activation):

    sentence_indices = Input(shape=input_shape, dtype=np.int32)
    emb_dim = 300  # embedding di 300 parole in italiano
    embedding_layer = pretrained_embedding_layer(word_to_vec_map, word_to_index, emb_dim)

    embeddings = embedding_layer(sentence_indices)   

    X = LSTM(256, return_sequences=True)(embeddings)
    X = Dropout(0.15)(X)
    X = LSTM(128)(X)
    X = Dropout(0.15)(X)
    X = Dense(num_activation, activation='softmax')(X)

    model = Model(sentence_indices, X)

    sequentialModel = Sequential(model.layers)    
    return sequentialModel

    model = Classifier((maxLen,), word_to_vec_map, word_to_index, maxLen, num_activation)
    ...
    model.fit(Y_train_indices, Z_train_oh, epochs=30, batch_size=32, shuffle=True)

    # attempt 1
    model.save('classificationTest.h5', True, True)
    modelRNN = load_model(r'C:\Users\Alessio\classificationTest.h5')

    # attempt 2
    model.save_weights("myWeight.h5")

    model = Classifier((maxLen,), word_to_vec_map, word_to_index, maxLen, num_activation)
    model.load_weights(r'C:\Users\Alessio\myWeight.h5') 

    # PREDICTION TEST
    code_train, category_train, category_code_train, text_train = read_csv_for_email(r'C:\Users\Alessio\Desktop\6Febbraio\2test.csv')

    categories, code_categories = get_categories(r'C:\Users\Alessio\Desktop\6Febbraio\2test.csv')

    X_my_sentences = text_train
    Y_my_labels = category_code_train
    X_test_indices = sentences_to_indices(X_my_sentences, word_to_index, maxLen)
    pred = model.predict(X_test_indices)

    def codeToCategory(categories, code_categories, current_code):

        i = 0;
        for code in code_categories:
            if code == current_code:
                return categories[i]
            i = i + 1 
        return "no_one_find"   

    # result
    for i in range(len(Y_my_labels)):
        num = np.argmax(pred[i])

    # Pretrained embedding layer
    def pretrained_embedding_layer(word_to_vec_map, word_to_index, emb_dim):
    """
    Creates a Keras Embedding() layer and loads in pre-trained GloVe 50-dimensional vectors.

    Arguments:
    word_to_vec_map -- dictionary mapping words to their GloVe vector representation.
    word_to_index -- dictionary mapping from words to their indices in the vocabulary (400,001 words)

    Returns:
    embedding_layer -- pretrained layer Keras instance
    """

    vocab_len = len(word_to_index) + 1                  # adding 1 to fit Keras embedding (requirement)

    ### START CODE HERE ###
    # Initialize the embedding matrix as a numpy array of zeros of shape (vocab_len, dimensions of word vectors = emb_dim)
    emb_matrix = np.zeros((vocab_len, emb_dim))

    # Set each row "index" of the embedding matrix to be the word vector representation of the "index"th word of the vocabulary
    for word, index in word_to_index.items():
        emb_matrix[index, :] = word_to_vec_map[word]

    # Define Keras embedding layer with the correct output/input sizes, make it trainable. Use Embedding(...). Make sure to set trainable=False. 
    embedding_layer = Embedding(vocab_len, emb_dim)
    ### END CODE HERE ###

    # Build the embedding layer, it is required before setting the weights of the embedding layer. Do not modify the "None".
    embedding_layer.build((None,))

    # Set the weights of the embedding layer to the embedding matrix. Your layer is now pretrained.
    embedding_layer.set_weights([emb_matrix])

    return embedding_layer

Do you have any kind of suggestion?

Thanks in Advance.

Edit1: if use the code of saving and loading in the same "page" (I'm using notebook jupyter) it works fine. If I change "page" it doesn't work. Could it be that there is something related with the tensorflow session?

Edit2: my final goal is to load a model, trained in Keras, with Deeplearning4J in java. So if you know a solution for "transforming" the keras model in something else readable in DL4J it will help anyway.

Edit3: add function pretrained_embedding_layer()

Edit4: dictionaries from word2Vec model read with gensim

from gensim.models import Word2Vec
model = Word2Vec.load('C:/Users/Alessio/Desktop/emoji_ita/embedding/glove_WIKI')

def getMyModels (model):
word_to_index = dict({})
index_to_word = dict({})
word_to_vec_map = dict({})
for idx, key in enumerate(model.wv.vocab):
    word_to_index[key] = idx
    index_to_word[idx] = key
    word_to_vec_map[key] = model.wv[key]
return word_to_index, index_to_word, word_to_vec_map

Answer 1

Are you pre-processing your data in the same way when you load your model ?

And if yes, did you set the seed of your pre-processing functions ? If you build a dictionnary with keras, are the sentences coming in the same order ?

Answer 2

I had the same problem before, so here is how you solve it. After making sure that the weights and summary are the same, try to print your random seed and check. If its value is changing from a session to another and if you tried tensorflow's seed, it means you need to disable the PYTHONHASHSEED environment variable. You can read more about it here: https://docs.python.org/3/using/cmdline.html#envvar-PYTHONHASHSEED

To disable it, go to your system environment variables and add PYTHONHASHSEED as a new variable if it doesn't exist. Then, set its value to 0 to disable it. Please note that it was done in this way because it has to be disabled before running the interpreter.