Checkpoint deep learning models in Keras

Question

I need help in implementing the checkpoint function in Keras. I am going to train a large dataset so in order to do that, first I trained a model using the iris flower dataset : http://machinelearningmastery.com/multi-class-classification-tutorial-keras-deep-learning-library/

since my own dataset is a lot similar to it only the difference is that my dataset is more bigger.

For the checkpoint function : http://machinelearningmastery.com/check-point-deep-learning-models-keras/

I understand the example using pima-indians dataset. Now I am trying to implement the same checkpoint function in the iris-flower script. Here is what I tried so far.

import numpy
from pandas import *
from keras.models import Sequential
from keras.layers import Dense
from keras.wrappers.scikit_learn import KerasClassifier
from keras.utils import np_utils
from sklearn.model_selection import cross_val_score, KFold
from sklearn.preprocessing import LabelEncoder
from sklearn.pipeline import Pipeline
from keras.callbacks import ModelCheckpoint

seed = 7
numpy.random.seed(seed)

dataframe = read_csv("iris.csv", header=None)
dataset = dataframe.values
X = dataset[:,0:4].astype(float)
Y = dataset[:,4]

# encode class value as integers
encoder = LabelEncoder()
encoder.fit(Y)
encoded_Y = encoder.transform(Y)
dummy_y = np_utils.to_categorical(encoded_Y)

def baseline_model():
    model = Sequential()
    model.add(Dense(4, input_dim=4, init='normal', activation='relu'))
    model.add(Dense(3, init='normal', activation='sigmoid'))
    model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
    return model

filepath="weights-improvement-{epoch:02d}-{val_acc:.2f}.hdf5"
checkpoint = ModelCheckpoint(filepath, monitor='val_acc', verbose=1, save_best_only=True, mode='max')
callbacks_list = [checkpoint]

estimator = KerasClassifier(build_fn=baseline_model, validation_split=0.33, nb_epoch=200, batch_size=5, callbacks=callbacks_list, verbose=0)
kfold = KFold(n_splits=10, shuffle=True, random_state=seed)
results = cross_val_score(estimator, X, dummy_y, cv=kfold)
print("Baseline: %.2f%% (%.2f%%)" % (results.mean()*100, results.std()*100))

This script produced the following error. I don't know how to troubleshoot it or maybe my arrangement in the script is wrong.

RuntimeError: Cannot clone object <keras.wrappers.scikit_learn.KerasClassifier object at 0x10e120fd0>, as the constructor does not seem to set parameter callbacks

I hope somebody can help me with this. Thank you.

Answer 1

I think that the problem is that your baseline_model() function is not returning the model it is creating; it should be something like:

def baseline_model():
    model = Sequential()
    model.add(Dense(4, input_dim=4, init='normal', activation='relu'))
    model.add(Dense(3, init='normal', activation='sigmoid'))
    model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
    return model

Answer 2

Instead of KerasClassifier, use the model itself.

model = baseline_model()
#declare your callback methods here
model.fit(x,y, batch_size=32, verbose=0, epochs=10, shuffle=True, validation_split = 0.1, callbacks = <your list of callbacks>)

Answer 3

I had the same error but with setting the 'units' parameter in the NN layer.

RuntimeError: Cannot clone object <tensorflow.python.keras.wrappers.scikit_learn.KerasClassifier object at 0x1496b97f0>, as the constructor either does not set or modifies parameter units

Downgrading scikit-learn from 0.23.1 to 0.21.2 solved the issue for me.

Checkout this issue on github: Link