How to train a CNN regression model, which takes multiple input images and gives a single value as output.?

Question

Some insights about my dataset. I have few videos, each of 60 seconds. For each second I have some value assigned to it in a csv file. So for 60 seconds I have 1800 frames(counting as per 30fps). so processing 30 images at once should give me an output(I need to train on all these 30 frames as the output actually depends on how much the values in the frames are changing).
I tried to tile all the 30 images in one single image and then trained a CNN model with that. It didn't give satisfactory results. Now the data format is basically (540,30,64,64) where the images are of size 64 by 64 pixels. I tried the below code, by giving input share as 30,64,64,1 but it gives the error as shown below. Please help me in this matter. Thanking you in advance.

from keras.layers.convolutional import Convolution2D,MaxPooling2D
model = Sequential()
#model.add(Input((240,100,1)))
model.add(Convolution2D(32,3,padding='same',activation='relu',input_shape=(30,64,64,1)))
model.add(MaxPooling2D(pool_size=(2,2)))
model.add(Flatten())
model.add(Dense(units = 100,activation = 'relu'))
model.add(Dense(units = 1,activation = 'softmax'))

The error call here is as follows

ValueError                                Traceback (most recent call last)
<ipython-input-10-f60da85accc3> in <module>()
      4 #model.add(Input((240,100,1)))
      5 model.add(Convolution2D(32,3,padding='same',activation='relu',input_shape=(30,64,64,1)))
----> 6 model.add(MaxPooling2D(pool_size=(2,2)))
      7 model.add(Flatten())
      8 model.add(Dense(units = 100,activation = 'relu'))

7 frames
/usr/local/lib/python3.7/dist-packages/tensorflow/python/training/tracking/base.py in _method_wrapper(self, *args, **kwargs)
    515     self._self_setattr_tracking = False  # pylint: disable=protected-access
    516     try:
--> 517       result = method(self, *args, **kwargs)
    518     finally:
    519       self._self_setattr_tracking = previous_value  # pylint: disable=protected-access

/usr/local/lib/python3.7/dist-packages/tensorflow/python/keras/engine/sequential.py in add(self, layer)
    221       # If the model is being built continuously on top of an input layer:
    222       # refresh its output.
--> 223       output_tensor = layer(self.outputs[0])
    224       if len(nest.flatten(output_tensor)) != 1:
    225         raise ValueError(SINGLE_LAYER_OUTPUT_ERROR_MSG)

/usr/local/lib/python3.7/dist-packages/tensorflow/python/keras/engine/base_layer.py in __call__(self, *args, **kwargs)
    950     if _in_functional_construction_mode(self, inputs, args, kwargs, input_list):
    951       return self._functional_construction_call(inputs, args, kwargs,
--> 952                                                 input_list)
    953 
    954     # Maintains info about the `Layer.call` stack.

/usr/local/lib/python3.7/dist-packages/tensorflow/python/keras/engine/base_layer.py in _functional_construction_call(self, inputs, args, kwargs, input_list)
   1089         # Check input assumptions set after layer building, e.g. input shape.
   1090         outputs = self._keras_tensor_symbolic_call(
-> 1091             inputs, input_masks, args, kwargs)
   1092 
   1093         if outputs is None:

/usr/local/lib/python3.7/dist-packages/tensorflow/python/keras/engine/base_layer.py in _keras_tensor_symbolic_call(self, inputs, input_masks, args, kwargs)
    820       return nest.map_structure(keras_tensor.KerasTensor, output_signature)
    821     else:
--> 822       return self._infer_output_signature(inputs, args, kwargs, input_masks)
    823 
    824   def _infer_output_signature(self, inputs, args, kwargs, input_masks):

/usr/local/lib/python3.7/dist-packages/tensorflow/python/keras/engine/base_layer.py in _infer_output_signature(self, inputs, args, kwargs, input_masks)
    860           # overridden).
    861           # TODO(kaftan): do we maybe_build here, or have we already done it?
--> 862           self._maybe_build(inputs)
    863           outputs = call_fn(inputs, *args, **kwargs)
    864 

/usr/local/lib/python3.7/dist-packages/tensorflow/python/keras/engine/base_layer.py in _maybe_build(self, inputs)
   2683     if not self.built:
   2684       input_spec.assert_input_compatibility(
-> 2685           self.input_spec, inputs, self.name)
   2686       input_list = nest.flatten(inputs)
   2687       if input_list and self._dtype_policy.compute_dtype is None:

/usr/local/lib/python3.7/dist-packages/tensorflow/python/keras/engine/input_spec.py in assert_input_compatibility(input_spec, inputs, layer_name)
    221                          'expected ndim=' + str(spec.ndim) + ', found ndim=' +
    222                          str(ndim) + '. Full shape received: ' +
--> 223                          str(tuple(shape)))
    224     if spec.max_ndim is not None:
    225       ndim = x.shape.rank

ValueError: Input 0 of layer max_pooling2d is incompatible with the layer: expected ndim=4, found ndim=5. Full shape received: (None, 30, 64, 64, 32)

I am using google colab for this purpose so it has tensorflow version 2.4.1 on it.

Answer 1

Issue is with this line

model.add(Convolution2D(32,3,padding='same',activation='relu',input_shape=(30,64,64,1)))

Generally Conv2D expects input of shape 4D. input_shape argument in Conv2D will not include batch_size In conv2D your input_shape should be

model.add(Convolution2D(32,3,padding='same',activation='relu',input_shape=(64,64,1)))

Output of Conv2D is 4+D tensor.

max_pooling2d layer expects input 4D tensor.