Passing images to Keras' CNN model with 6 channels

Question

I'm having trouble with Keras imagedatagenerator class. I want to pass it an image with 6 channels to train background subtraction - I have the image stacked depthwise with its background, which is calculated using SubSense algorithm. This gives an image of shape (X,X,6) instead of two images with (X,X,3)

But opencv doesn't really let you save this so I had to save it as a numpy file. Now I want to get that numpy file into Keras as an image for training a CNN that will eventually return just the foreground masks but obviously trying to run it through ImageDataGenerator returns 0 images because it has too many channels.

My code is:

all_files = os.listdir("null/train")
arr=[]

for i in all_files:
    file_path= "null/train/" + i
    X = np.load(file_path)
    arr.append (X)

datagen = ImageDataGenerator(
    featurewise_center=True,
    featurewise_std_normalization=True,
    rotation_range=20,
    width_shift_range=0.2,
    height_shift_range=0.2,
    horizontal_flip=True)

datagen.fit(arr)

The resulting Error was:

null/anaconda3\\envs\\tensorflow\\lib\\site-packages\\keras_preprocessing\\image\\image_data_generator.py:940: UserWarning: Expected input to be images (as Numpy array) following the data format convention "channels_last" (channels on axis 3), ie expected either 1, 3 or 4 channels on axis 3. However, it was passed an array with shape (451, 321, 321, 6) (6 channels). ' channels).')

Is there a way to force it to take numpy files with 6 channels?

Answer 1

for your case, you need to modify the preprocessing_function which reads your rank 3 image as input. whatever you are doing to make it 6 channel image, do that change inside that function. please see an example (I'm passing a 3 channel image in ImageGenerator).

see below, arr is 3 channel image.I write a preProsFunc and assigned it to preprocessing_function of ImageGenerator. I am just concatenating so as to make 6 channel image. similarly you can write your logic inside proprocessing_function

import tensorflow as tf
import numpy as np

arr = np.zeros((4,4,3))
arr = np.expand_dims(arr, axis=0)

def preProsFunc(arr):
  arr1 = np.zeros((4,4,3))
  arr1 = np.expand_dims(arr1, axis=0)
  return np.concatenate((arr, arr1), axis=2)

datagen = tf.keras.preprocessing.image.ImageDataGenerator(
    featurewise_center=True,
    featurewise_std_normalization=True,
    rotation_range=20,
    width_shift_range=0.2,
    height_shift_range=0.2,
    horizontal_flip=True,
    preprocessing_function=preProsFunc
    )

datagen.fit(arr)

let me know for anything.