Pass input to CNN iteratively, one image array at a time

Question

Is there any possible way to pass the input to a CNN one array at a time rather than making an array of all the arrays?

From all the online examples and tutorials, the CNN input is given as:

X.append(np.array(cv2.resize(cv2.imread(list_of_images+'/'+image), (224,224), interpolation=cv2.INTER_CUBIC)))

where x is a 4-d array containing all 3d arrays, which is then fed to the model as:

model.fit(X, Y, batch_size=128, epochs=25)

Is there a way to pass the array iteratively, for a large dataset as python has an upper limit to the numpy array size.

Answer 1

I found the solution for my query from the link: https://www.kaggle.com/rejpalcz/cnn-128x128x4-keras-from-scratch-lb-0-328

Look at the class: 'ProteinDataGenerator'