每次使用 Keras 開始新的訓練 session 時，如何防止丟失和准確性重新開始？

Question

我正在學習如何通過在 Keras 中創建 CNN 來構建圖像分類器。 每次我開始新的培訓 session 時，如何防止我的程序重新開始。 代碼如下，包括我嘗試過的一些注釋掉的代碼。 謝謝！

我試圖創建檢查點，但准確性和損失似乎仍然重置。 我也嘗試過 model.save() 和 model.save_weights() 然后將它們加載到新的 model 中，但我的准確性和損失似乎仍然從頭開始。

                                 IMPORTS 
# import CIFAR10 data
from keras.datasets import cifar10

# import keras utils
import keras.utils as utils

# import Sequential model
from keras.models import Sequential

# import layers
from keras.layers import Dense, Dropout, Flatten
from keras.layers.convolutional import Conv2D, MaxPooling2D

# normalizes values in kernal
from keras.constraints import maxnorm

# import compiler optimizers
from keras.optimizers import SGD

# import keras checkpoint
from keras.callbacks import ModelCheckpoint

# import h5py
import h5py

                             END IMPORTS 

# load cifar10 data
(x_train, y_train), (x_test, y_test) = cifar10.load_data()

# format trains and tests to float32 and divide by 255.0
x_train = x_train.astype('float32') / 255.0
x_test = x_test.astype('float32') / 255.0

# change y_train and y_test to utils categorical
y_train = utils.to_categorical(y_train)
y_test = utils.to_categorical(y_test)

# create labels array
labels = ['airplane', 'automobile', 'bird', 'cat', 'deer', 'dog', 'frog', 'horse', 'ship', 'truck']

                         SEQUENTIAL MODEL 1 

model = Sequential()

model.add(Conv2D(filters=32, kernel_size=(3, 3), input_shape=(32, 32, 3), activation='relu', padding='same',
                 kernel_constraint=maxnorm(3)))

#### add second convolution layer - MaxPooling2d ####
# decreases image size from 32x32 to 16x16
# pool_size: finds max value in each 2x2 section of input
model.add(MaxPooling2D(pool_size=(2, 2)))

#### flatten features ####
# converts matrix to a 1 dimensional array
model.add(Flatten())

#### add third convolution layer - first Dense and feed into it ####
# creates prediction network
# units: 512 neurons for first layer
# activation: relu for accuracy
# kernal_constraint: maxnorm
model.add(Dense(units=512, activation='relu', kernel_constraint=maxnorm(3)))

#### add fourth convolution later - Dropout - kills some neurons - prevents overfitting - TRAINING ONLY ####
# improves reliability
# rate: 0.5 means kill half the neurons
# only to be used while training
model.add(Dropout(rate=0.5))

#### add fifth convolution layer - Second Dense layer - Creates 10 outputs because we have 10 categories ####
# produces output for each of the 10 categories
# units: 10 categories = 10 output units
# activation = 'softmax' because we are calculating the probabilities of each of the 10 categories (floats)
model.add(Dense(units=10, activation='softmax'))

############################## END SEQUENTIAL MODEL ##########################                 

############################## COMPILER ######################################
model.compile(optimizer=SGD(lr=0.01), loss='categorical_crossentropy', metrics=['accuracy'])

################################ END COMPILER ################################

################################ SAVE DATA ###################################

# saves the training data
model.save(filepath='model.h5')

# create model checkpoint based on best accuracy
#filepath = 'model.h5'
#checkpoint = ModelCheckpoint(filepath, monitor='val_accuracy', save_best_only='True',
                             #save_weights_only='False', mode='max', period=1)

#callbacks_list = [checkpoint]

# save weights
model.save_weights('model_weights.h5')

############################### END SAVE DATA ################################
model.fit(x=x_train, y=y_train, validation_split=0.1, epochs=20, batch_size=32, shuffle='True')

Answer 1

你的問題是你在訓練之前保存 model 。 您必須首先安裝進行訓練的 model，然后保存 model。 還附上更改的代碼

from keras.datasets import cifar10

# import keras utils
import keras.utils as utils

# import Sequential model
from keras.models import Sequential

# import layers
from keras.layers import Dense, Dropout, Flatten
from keras.layers.convolutional import Conv2D, MaxPooling2D

# normalizes values in kernal
from keras.constraints import maxnorm

# import compiler optimizers
from keras.optimizers import SGD

# import keras checkpoint
from keras.callbacks import ModelCheckpoint

# import h5py
import h5py



# load cifar10 data
(x_train, y_train), (x_test, y_test) = cifar10.load_data()

# format trains and tests to float32 and divide by 255.0
x_train = x_train.astype('float32') / 255.0
x_test = x_test.astype('float32') / 255.0

# change y_train and y_test to utils categorical
y_train = utils.to_categorical(y_train)
y_test = utils.to_categorical(y_test)

# create labels array
labels = ['airplane', 'automobile', 'bird', 'cat', 'deer', 'dog', 'frog', 'horse', 'ship', 'truck']



model = Sequential()

model.add(Conv2D(filters=32, kernel_size=(3, 3), input_shape=(32, 32, 3), activation='relu', padding='same',
                 kernel_constraint=maxnorm(3)))

#### add second convolution layer - MaxPooling2d ####
# decreases image size from 32x32 to 16x16
# pool_size: finds max value in each 2x2 section of input
model.add(MaxPooling2D(pool_size=(2, 2)))

#### flatten features ####
# converts matrix to a 1 dimensional array
model.add(Flatten())

#### add third convolution layer - first Dense and feed into it ####
# creates prediction network
# units: 512 neurons for first layer
# activation: relu for accuracy
# kernal_constraint: maxnorm
model.add(Dense(units=512, activation='relu', kernel_constraint=maxnorm(3)))

#### add fourth convolution later - Dropout - kills some neurons - prevents overfitting - TRAINING ONLY ####
# improves reliability
# rate: 0.5 means kill half the neurons
# only to be used while training
model.add(Dropout(rate=0.5))

#### add fifth convolution layer - Second Dense layer - Creates 10 outputs because we have 10 categories ####
# produces output for each of the 10 categories
# units: 10 categories = 10 output units
# activation = 'softmax' because we are calculating the probabilities of each of the 10 categories (floats)
model.add(Dense(units=10, activation='softmax'))

############################## END SEQUENTIAL MODEL ##########################                 

############################## COMPILER ######################################
model.compile(optimizer=SGD(lr=0.01), loss='categorical_crossentropy', metrics=['accuracy'])


model.fit(x=x_train, y=y_train, validation_split=0.1, epochs=20, batch_size=32, shuffle='True')

################################ END COMPILER ################################

################################ SAVE DATA ###################################

# saves the training data
model.save(filepath='model.h5')

# create model checkpoint based on best accuracy
#filepath = 'model.h5'
#checkpoint = ModelCheckpoint(filepath, monitor='val_accuracy', save_best_only='True',
                             #save_weights_only='False', mode='max', period=1)

#callbacks_list = [checkpoint]

# save weights
model.save_weights('model_weights.h5')

############################### END SAVE DATA ################################

讓我知道這個是否奏效

Answer 2

您應該在訓練后保存 model 並使用keras.models.load_model 。

請參閱以下代碼段。

# import CIFAR10 data
from keras.datasets import cifar10

# import keras utils
import keras.utils as utils

# import Sequential model
from keras.models import Sequential

# import layers
from keras.layers import Dense, Dropout, Flatten
from keras.layers.convolutional import Conv2D, MaxPooling2D

# normalizes values in kernal
from keras.constraints import maxnorm

# import compiler optimizers
from keras.optimizers import SGD

# import keras checkpoint
from keras.callbacks import ModelCheckpoint

# import h5py
import h5py


from keras.models import load_model

# load cifar10 data
(x_train, y_train), (x_test, y_test) = cifar10.load_data()

# format trains and tests to float32 and divide by 255.0
x_train = x_train.astype('float32') / 255.0
x_test = x_test.astype('float32') / 255.0

# change y_train and y_test to utils categorical
y_train = utils.to_categorical(y_train)
y_test = utils.to_categorical(y_test)

# create labels array
labels = ['airplane', 'automobile', 'bird', 'cat', 'deer', 'dog', 'frog', 'horse', 'ship', 'truck']


model = Sequential()

model.add(Conv2D(filters=32, kernel_size=(3, 3), input_shape=(32, 32, 3), activation='relu', padding='same',
                 kernel_constraint=maxnorm(3)))

#### add second convolution layer - MaxPooling2d ####
# decreases image size from 32x32 to 16x16
# pool_size: finds max value in each 2x2 section of input
model.add(MaxPooling2D(pool_size=(2, 2)))

#### flatten features ####
# converts matrix to a 1 dimensional array
model.add(Flatten())

#### add third convolution layer - first Dense and feed into it ####
# creates prediction network
# units: 512 neurons for first layer
# activation: relu for accuracy
# kernal_constraint: maxnorm
model.add(Dense(units=512, activation='relu', kernel_constraint=maxnorm(3)))

#### add fourth convolution later - Dropout - kills some neurons - prevents overfitting - TRAINING ONLY ####
# improves reliability
# rate: 0.5 means kill half the neurons
# only to be used while training
model.add(Dropout(rate=0.5))

#### add fifth convolution layer - Second Dense layer - Creates 10 outputs because we have 10 categories ####
# produces output for each of the 10 categories
# units: 10 categories = 10 output units
# activation = 'softmax' because we are calculating the probabilities of each of the 10 categories (floats)
model.add(Dense(units=10, activation='softmax'))

############################## END SEQUENTIAL MODEL ##########################                 

############################## COMPILER ######################################
model.compile(optimizer=SGD(lr=0.01), loss='categorical_crossentropy', metrics=['accuracy'])

################################ END COMPILER ################################

################################ SAVE DATA ###################################

model = load_model('model.h5')

# create model checkpoint based on best accuracy
#filepath = 'model.h5'
#checkpoint = ModelCheckpoint(filepath, monitor='val_accuracy', save_best_only='True',
                             #save_weights_only='False', mode='max', period=1)

#callbacks_list = [checkpoint]

# # save weights
# model.save_weights('model_weights.h5')

############################### END SAVE DATA ################################
model.fit(x=x_train, y=y_train, validation_split=0.1, epochs=1, batch_size=32, shuffle='True')

# saves the training data
model.save(filepath='model.h5')

加載保存的 model 並重新訓練后，損失和准確度將從先前停止的值開始。

44800/45000 [=============================>.] - ETA：0s - 損失：1.9399 - acc：0.3044832/45000 [ ============================>。] - ETA：0s - 損失：1.9398 - acc：0.3044864/45000 [==== =======================>。] - ETA：0s - 損失：1.9397 - acc：0.3044896/45000 [======== ====================>。] - ETA：0s - 損失：1.9396 - acc：0.3044928/45000 [============ ================>。] - ETA：0s - 損失：1.9397 - acc：0.3044960/45000 [================ ============>。] - ETA：0s - 損失：1.9397 -acc：0.3044992/45000 [==================== ========>。] - ETA：0s - 損失：1.9395 - acc：0.3045000/45000 [======================= ======] - 82s 2ms/步 - 損失：1.9395 - acc：0.3030 - val_loss：1.7316 - val_acc：0.3852

在接下來的運行中，

紀元 1/1 32/45000 [.......................] - ETA：3:13 - 損失：1.7473 -加速：0. 64/45000 [.......................] - ETA：2:15 - 損失：1.7321 -加速：0. 96/45000 [.......................] - ETA：1:58 - 損失：1.6830 -加速：0. 128/45000 [.......................] - ETA：1:48 - 損失：1.6729 -加速：0. 160/45000 [.......................] - ETA：1:41 - 損失：1.6876 -累積：0。

但是，請注意，當您從文件加載 model 時，不需要編譯 model。

Answer 3

據我了解，如果您關閉培訓 session，您的問題是繼續培訓。

這里有一些有用的資源來解決您的問題。

• 從走向數據科學
• 從 SO
• 來自 pyimagesearch

一般來說，您可以使用 keras 檢查來源、檢查點和恢復訓練。

希望它解決。