Keras：不能使用 ImageDataGenerator 作为 model.fit 中的验证数据

Question

I know that sequential models can use dataset iterators as validation data from the documentation, see https://keras.io/models/sequential/ :

validation_data : Data on which to evaluate the loss and any model metrics at the end of each epoch. The model will not be trained on this data. validation_data will override validation_split. validation_data could be: - tuple (x_val, y_val) of Numpy arrays or tensors - tuple (x_val, y_val, val_sample_weights) of Numpy arrays - dataset or a dataset iterator

However, feeding a dataset iterator built from complete data to this parameter raises an error in the file keras/engine/training.py of the library, lines 1158 to 1170, since they check sizes in the file and that type of generator generates a list of batches. What did I miss?

The minimalist code, taken from https://keras.io/preprocessing/image/ and https://keras.io/examples/cifar10_cnn/ , pass the model definition and focus on model.fit -type lines:

import keras
from keras.datasets import cifar10
from keras.utils import np_utils
from keras.preprocessing.image import ImageDataGenerator
from keras.models import Sequential
from keras.layers import Dense, Dropout, Activation, Flatten
from keras.layers import Conv2D, MaxPooling2D


def def_model(input_shape, n_classes):
    """Stolen from [here](https://keras.io/examples/cifar10_cnn/)."""
    model = Sequential()
    model.add(Conv2D(32, (3, 3), padding='same', input_shape=input_shape[1:]))
    model.add(Activation('relu'))
    model.add(Conv2D(32, (3, 3)))
    model.add(Activation('relu'))
    model.add(MaxPooling2D(pool_size=(2, 2)))
    model.add(Dropout(0.25))

    model.add(Conv2D(64, (3, 3), padding='same'))
    model.add(Activation('relu'))
    model.add(Conv2D(64, (3, 3)))
    model.add(Activation('relu'))
    model.add(MaxPooling2D(pool_size=(2, 2)))
    model.add(Dropout(0.25))

    model.add(Flatten())
    model.add(Dense(512))
    model.add(Activation('relu'))
    model.add(Dropout(0.5))
    model.add(Dense(n_classes))
    model.add(Activation('softmax'))

    # initiate RMSprop optimizer
    opt = keras.optimizers.RMSprop(learning_rate=0.0001, decay=1e-6)

    # Let's train the model using RMSprop
    model.compile(loss='categorical_crossentropy',
                  optimizer=opt,
                  metrics=['accuracy'])

    return model


def main():
    num_classes = 10
    epochs = 1

    (x_train, y_train), (x_test, y_test) = cifar10.load_data()
    y_train = np_utils.to_categorical(y_train, num_classes)
    y_test = np_utils.to_categorical(y_test, num_classes)

    datagen = ImageDataGenerator(
        featurewise_center=True,
        featurewise_std_normalization=True,
        rotation_range=20,
        width_shift_range=0.2,
        height_shift_range=0.2,
        horizontal_flip=True)

    datagen_test = ImageDataGenerator(
        featurewise_center=True,
        featurewise_std_normalization=True)

    # compute quantities required for featurewise normalization
    # (std, mean, and principal components if ZCA whitening is applied)
    datagen.fit(x_train)
    datagen_test.fit(x_train)

    model = def_model(x_train.shape, num_classes)

    # fits the model on batches with real-time data augmentation:
    model.fit_generator(datagen.flow(x_train, y_train, batch_size=32),
                        steps_per_epoch=1, epochs=epochs)  # len(x_train) / 32

    # here's a more "manual" example
    for e in range(epochs):
        print('Epoch', e)
        batches = 0
        for x_batch, y_batch in datagen.flow(x_train, y_train, batch_size=32):
            model.fit(x_batch, y_batch, validation_data=datagen_test.flow(
                x_train, y_train, batch_size=32))
            batches += 1
            if batches >= len(x_train) / 32:
                # we need to break the loop by hand because
                # the generator loops indefinitely
                break


if __name__ == "__main__":

My version of the libraries:

$ pip freeze | grep Keras
Keras==2.3.1
Keras-Applications==1.0.8
Keras-Preprocessing==1.1.0

The full output of running the code:

Using TensorFlow backend.
2019-11-02 09:17:11.946893: I tensorflow/core/platform/cpu_feature_guard.cc:145] This TensorFlow binary is optimized with Intel(R) MKL-DNN to use the following CPU instructions in performance critical operations:  SSE4.1 SSE4.2 AVX AVX2 FMA
To enable them in non-MKL-DNN operations, rebuild TensorFlow with the appropriate compiler flags.
2019-11-02 09:17:12.111959: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 3000000000 Hz
2019-11-02 09:17:12.112625: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x55a874c0e510 executing computations on platform Host. Devices:
2019-11-02 09:17:12.112676: I tensorflow/compiler/xla/service/service.cc:175]   StreamExecutor device (0): Host, Default Version
2019-11-02 09:17:12.116662: I tensorflow/core/common_runtime/process_util.cc:115] Creating new thread pool with default inter op setting: 2. Tune using inter_op_parallelism_threads for best performance.
Epoch 1/1
1/1 [==============================] - 2s 2s/step - loss: 2.3134 - accuracy: 0.0312
Epoch 0
Traceback (most recent call last):
  File "tmp.py", line 90, in <module>
    main()
  File "tmp.py", line 81, in main
    x_train, y_train, batch_size=32))
  File "/home/robin/anaconda3/lib/python3.7/site-packages/keras/engine/training.py", line 1170, in fit
    len(validation_data))
ValueError: When passing validation_data, it must contain 2 (x_val, y_val) or 3 (x_val, y_val, val_sample_weights) items, however it contains 1563 items

Answer 1

My code was a bit broken, since I used two times the fit method (either fit_generator or fit ), see below. The code runs when I put the validation_data parameter in fit_generator instead of fit . The documentation of Keras however stated that one can use fit with a dataset iterator, which must be different than a generator.

    model.fit_generator(datagen.flow(x_train, y_train, batch_size=32),
                        steps_per_epoch=1, epochs=epochs)  # len(x_train) / 32

    # here's a more "manual" example
    for e in range(epochs):
        print('Epoch', e)
        batches = 0
        for x_batch, y_batch in datagen.flow(x_train, y_train, batch_size=32):
            model.fit(x_batch, y_batch, validation_data=datagen_test.flow(
                x_train, y_train, batch_size=32))

Thanks to @natthaphon-hongcharoen for their hint.