Keras custom data generator - Error: 'int' object has no attribute 'shape'

Question

I am using tensorflow.keras with Tensorflow version 2.4.1. I have written a custom generator but during startig of first epoch it gives error: 'int' object has no attribute 'shape'

def data_generator(path, model_path):
    loadmodel = load_model(model_path)
    new_model = Model(loadmodel.input, loadmodel.layers[-2].output)
    dirs = os.listdir(path)
    dirs = np.sort(dirs)
    while True:
        for i in range(len(dirs)):
            print("Directory: ", os.path.join(path, dirs[i]))
            vid_list = os.listdir(os.path.join(path, dirs[i]))
            for j in range(len(vid_list)):
                video = cv2.VideoCapture(os.path.join(path, dirs[i], vid_list[j]))
                cnt = 0
                x_train = np.zeros((10, 1024))
                while True:
                    ret, frame = video.read()
                    if not ret:
                        break
                    frame = process_image_for_video(frame, (299, 299, 3))
                    frame = np.expand_dims(frame, axis=0)
                    predictions = new_model.predict(frame)
                    x_train[cnt] = predictions
                    cnt = cnt + 1
                    if cnt == 10:
                        yield_val = (x_train, i)
                        yield yield_val
                        cnt = 0
                        x_train = np.zeros((10, 1024))

The error is:

Traceback (most recent call last):
  File "/mnt/Darshil/IIITB/Semester2/NC854DigitalImageProcessing/PaperReading/video_classification/archive/five-video-classification-methods-master/train_custom.py", line 60, in <module>
    vmodel.fit_generator(generator, epochs=epoch, steps_per_epoch=steps, verbose=1, validation_data=generator, validation_steps=steps)
  File "/home/darshil/.virtualenvs/ml/lib/python3.8/site-packages/tensorflow/python/keras/engine/training.py", line 1847, in fit_generator
    return self.fit(
  File "/home/darshil/.virtualenvs/ml/lib/python3.8/site-packages/tensorflow/python/keras/engine/training.py", line 1050, in fit
    data_handler = data_adapter.DataHandler(
  File "/home/darshil/.virtualenvs/ml/lib/python3.8/site-packages/tensorflow/python/keras/engine/data_adapter.py", line 1100, in __init__
    self._adapter = adapter_cls(
  File "/home/darshil/.virtualenvs/ml/lib/python3.8/site-packages/tensorflow/python/keras/engine/data_adapter.py", line 798, in __init__
    output_shapes = nest.map_structure(_get_dynamic_shape, peek)
  File "/home/darshil/.virtualenvs/ml/lib/python3.8/site-packages/tensorflow/python/util/nest.py", line 659, in map_structure
    structure[0], [func(*x) for x in entries],
  File "/home/darshil/.virtualenvs/ml/lib/python3.8/site-packages/tensorflow/python/util/nest.py", line 659, in <listcomp>
    structure[0], [func(*x) for x in entries],
  File "/home/darshil/.virtualenvs/ml/lib/python3.8/site-packages/tensorflow/python/keras/engine/data_adapter.py", line 792, in _get_dynamic_shape
    shape = t.shape
AttributeError: 'int' object has no attribute 'shape'

Answer 1

You're returning i as your target, which is an integer. You will need to transform i into a NumPy array. You know the drill:

i = np.array([i])

In this part of the code:

for i in range(len(dirs)):
  # ... 
  yield_val = (x_train, i)
  yield yield_val