Keras ValueError when loading weights

Question

This is the error message I got

Traceback (most recent call last):
    File "/home/xxx/Documents/program/test.py", line 27, in <module>
model.load_weights('models/model.h5')
    File "/home/xxx/Documents/program/venv/lib/python3.6/site-packages/tensorflow/python/keras/engine/network.py", line 1391, in load_weights
saving.load_weights_from_hdf5_group(f, self.layers)
    File "/home/xxx/Documents/program/venv/lib/python3.6/site-packages/tensorflow/python/keras/engine/saving.py", line 732, in load_weights_from_hdf5_group
' layers.')
ValueError: You are trying to load a weight file containing 2 layers into a model with 0 layers.

From this minimal example that produces the error

from tensorflow import keras
from data import get_data

X_train, y_train, X_val, y_val = get_data()  # get some train and val data

model = keras.Sequential()
model.add(keras.layers.Dense(64, activation='relu'))
model.add(keras.layers.Dense(7, activation='softmax'))

model.compile(
    optimizer=keras.optimizers.Adam(1e-4),
    loss='categorical_crossentropy',
    metrics=['accuracy']
)

model.fit(
    x=X_train,
    y=y_train,
    batch_size=500,
    epochs=200,
    verbose=2,
    validation_data=(X_val, y_val)
)

model.save_weights('models/model.h5')

model.load_weights('models/model.h5')

Directly running this does not produce the error. However, when I run the program for a second time commenting out the training part (from line 10 to 25) trying to load the weights, it gives me this error.

I am using Tensorflow 1.9.0 and the built-in Keras.

Answer 1

As mentioned above, there seems to be a bug in keras sequential mode: https://github.com/keras-team/keras/issues/10417 .

However, you can get around this by using the Keras Functional API (you'll also find the Functional API much more useful when you're building trickier RNNs models with complicated I/O and tensor concatenations).

The disadvantage of using model.save_weights() method to save your neural network is that you have to invoke the model architecture before you load .h5 weights into the NN. If you instead save the whole model (both parameters AND architecture), you'll find that it's much easier to load trained model into a Python object. You can achieve this by using model.save() method.

### TRAINING CODE
import tensorflow as tf
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
# some data
iris = load_iris()
X, y = iris.data, iris.target
X_train, X_val, y_train, y_val = train_test_split(X, y, test_size=0.2)
y_train_oh = tf.keras.utils.to_categorical(y_train)
y_val_oh = tf.keras.utils.to_categorical(y_val)

# Keras Functional API
x = tf.keras.Input(shape=(4,))
dense = tf.keras.layers.Dense(64, activation='relu')(x)
dense = tf.keras.layers.Dense(3, activation='softmax')(dense)
model = tf.keras.Model(inputs=x, outputs=dense)
model.compile(optimizer=tf.keras.optimizers.Adam(1e-4),
              loss='categorical_crossentropy',
              metrics=['accuracy'])
# training
model.fit(X_train, y_train_oh, 16, epochs=20, validation_data=(X_val, y_val_oh))
# save weights
model.save_weights('models/model_weights.h5')
# save weights AND architecture
model.save('models/model.h5')


### TESTING CODE
# Model loading using .h5 weights file
import tensorflow as tf
x = tf.keras.Input(shape=(4,))
dense = tf.keras.layers.Dense(64, activation='relu')(x)
dense = tf.keras.layers.Dense(3, activation='softmax')(dense)
model2 = tf.keras.Model(inputs=x, outputs=dense)
model2.load_weights('models/model_weights.h5')

# Model loading using .h5 model file
import tensorflow as tf
model3 = tf.keras.models.load_model('models/model.h5') # simpler API, but bigger filesize