如何保存Keras的培训历史以进行交叉验证（循环）？

Question

For a cross validation, how to save the training history of different training set and cross validation set? I thought 'a' appending mode of pickle write will works but actually it didn't work. If possible, could you please also instruct me the way to save all the models, right now I can only save the last trained model with model.save(file) .

historyfile = 'history.pickle'
f = open(historyfile,'w')
f.close()
ind = 0
save = {}
for train, test in kfold.split(input,output):
    ind = ind+1
    #create model
    model = model_FCN()
    # fit the model
    history = model.fit(input[list(train)], output[list(train)], batch_size = 16, epochs = 100, verbose =1, validation_data =(input[list(test)],output[list(test)]))
    #save to file 
    try:
        f = open(historyfile,'a') ## appending mode??
        save['cv'+ str(ind)]= history.history
        pickle.dump(save, f, pickle.HIGHEST_PROTOCOL)
        f.close()
    except Exception as e:
        print('Unable to save data to', historyfile, ':', e)

    scores = model.evaluate(MR_patch[list(test)], CT_patch[list(test)], verbose=0)
    print("%s: %.2f" % (model.metrics_names[1], scores[1]))
    cvscores.append(scores[1])
    print("cross validation stage: " + str(ind))

print("%.2f (+/- %.2f)" % (np.mean(cvscores), np.std(cvscores)))

Answer 1

To save model after each epoch for certain train and validate data you can use Callback :

For example:

from keras.callbacks import ModelCheckpoint
import os

output_directory = '' # here should be path to output directory    
model_checkpoint = ModelCheckpoint(os.path.join(output_directory , 'weights.{epoch:02d}-{val_loss:.2f}.hdf5'))
model.fit(input[list(train)],
          output[list(train)],
          batch_size=16,
          epochs=100,
          verbose=1,
          validation_data=(input[list(test)],output[list(test)]),
          callbacks=[model_checkpoint])

After each epoch your model will be saved in file. More information about this callback you can find in documentation ( https://keras.io/callbacks/ )

If you want to save model trained on each fold, you can simply add model.save(file) in your for loop:

model.fit(input[list(train)],
          output[list(train)],
          batch_size=16,
          epochs=100,
          verbose=1,
          validation_data=(input[list(test)],output[list(test)]))
model.save(os.path.join(output_directory, 'fold_{}_model.hdf5'.format(ind)))

To save history: You can save history once, without appending it to file on each loop. After for loop you should get dictionary with keys (marks of your folds) and values (history on each fold) and save this dictionary like this:

f = open(historyfile, 'wb')
pickle.dump(save, f, pickle.HIGHEST_PROTOCOL)
f.close()