ImageDataGenerator shape issues in Keras

Question

datagen = ImageDataGenerator(
            featurewise_center=False,  # set input mean to 0 over the dataset
            samplewise_center=False,  # set each sample mean to 0
            featurewise_std_normalization=False,  # divide inputs by std of the dataset
            samplewise_std_normalization=False,  # divide each input by its std
            zca_whitening=False,  # apply ZCA whitening
            rotation_range=15,  # randomly rotate images in the range (degrees, 0 to 180)
            width_shift_range=0.1,  # randomly shift images horizontally (fraction of total width)
            height_shift_range=0.1,  # randomly shift images vertically (fraction of total height)
            horizontal_flip=True,  # randomly flip images
            vertical_flip=False)  # randomly flip images
        # (std, mean, and principal components if ZCA whitening is applied).
        # datagen.fit(x_train)


        print(x_train.shape)

        def data_generator(generator, x, y1, y2, batch_size):
            genX = generator.flow(x, seed=7, batch_size=batch_size)
            genY1 = generator.flow(y1, seed=7, batch_size=batch_size)
            genY2 = generator.flow(y2, seed=7, batch_size=batch_size)
            while(True):
                Xi = genX.next()
                Yi1 = genY1.next()
                Yi2 = genY2.next()
                yield Xi, [Yi1, Yi2]

And this is how I'm calling model.fit_generator

model.fit_generator(data_generator(datagen, x_train, y_train, y_aux_train, params['batch_size']),
                            epochs=params['epochs'], steps_per_epoch=150,
                            validation_data=data_generator(datagen, x_test, y_test, y_aux_test, params['batch_size']), 
                            validation_steps=100, callbacks=[reduce_lr, tensorboard],verbose=2)

This is the error I get -

ValueError: ('Input data in NumpyArrayIterator should have rank 4. You passed an array with shape', (5630, 4))

Answer 1

What are your x , y1 and y2 ? The input data to ImageDataGenerator must have 4 dimensions (batch, channel, height, width). Your data is something completely different, that is why you are getting the error.

UPDATE:

According to the docs :

flow(x, y=None, batch_size=32, shuffle=True, sample_weight=None, seed=None, save_to_dir=None, save_prefix='', save_format='png', subset=None)

x: Input data. Numpy array of rank 4 or a tuple. If tuple, the first element should contain the images and the second element another numpy array or a list of numpy arrays that gets passed to the output without any modifications. Can be used to feed the model miscellaneous data along with the images.

In genY1 = generator.flow(y1, seed=7, batch_size=batch_size) you pass your labels (that are of shape (4,) as I can see) as features, and ImageDataGenerator expects them to have 4 dimensions.

You should not pass your labels like this. Try instead something like this:

datagen.flow((x_train, [y_train, y_aux_train]), batch_size=params['batch_size'])

Or:

datagen.flow(x=x_train, y=[y_train, y_aux_train], batch_size=params['batch_size'])

Answer 2

I don't know the shape of the input you are giving... But from the error, I can tell you the following... Yes, your input value for parameter x must be of rank 4... I'll explain to you with an example. MNIST dataset when downloaded from Keras.datasets they come as NumPy ndarray of shape 60000(for test), 28,28 if you pass this to ImageDataGenerator as an input for the parameter, 'x' you get the same error. You can simply overcome that by resizing your array.

<your image ndarray>.reshape(<your image ndarray>.shape[0],28,28,1)

NOTE: The last value in the above line ie 1 for grayscale images and 3 for RGB images respectively which is the number of color channels in your image.

Therefore I suggest you resize your ndarray such that it has four dimensions.