Keras negative dimension size Conv2D

Question

I have been playing around with the kernel sizes and channel arrangements for a while with no luck. I am not entirely sure of how to calculate the corrector parameters for the Conv2D layers and I am unsure how much changing these parameters will affect the similarity to the model in the paper.

Any help would be greatly appreciated.

The model that I attempting to build based on the design in the literature

input_shape = (4, 30, 180)
model = Sequential()
model.add(Convolution2D(32, (8, 8), strides=(4,4), activation='relu', input_shape=(4,30,180), data_format='channels_first'))
model.add(Activation('relu'))
model.add(Convolution2D(64, (4, 4), strides=(2, 2)))
model.add(Activation('relu'))
model.add(Convolution2D(64, (3, 3), strides=(1, 1)))
model.add(Activation('relu'))
model.add(Flatten())
model.add(Dense(512))
model.add(Activation('relu'))
model.add(Dense(2))
model.add(Activation('linear'))

The error messages that I recieved

Traceback (most recent call last):
  File "/Users/zacharyfrederick/opt/anaconda3/lib/python3.7/site-packages/tensorflow/python/framework/ops.py", line 1659, in _create_c_op
    c_op = c_api.TF_FinishOperation(op_desc)
tensorflow.python.framework.errors_impl.InvalidArgumentError: Negative dimension size caused by subtracting 3 from 2 for 'conv2d_3/convolution' (op: 'Conv2D') with input shapes: [?,15,2,64], [3,3,64,64].

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "stock_env.py", line 101, in <module>
    model.add(Convolution2D(64, (3, 3), strides=(1, 1)))
  File "/Users/zacharyfrederick/opt/anaconda3/lib/python3.7/site-packages/keras/engine/sequential.py", line 181, in add
    output_tensor = layer(self.outputs[0])
  File "/Users/zacharyfrederick/opt/anaconda3/lib/python3.7/site-packages/keras/engine/base_layer.py", line 457, in __call__
    output = self.call(inputs, **kwargs)
  File "/Users/zacharyfrederick/opt/anaconda3/lib/python3.7/site-packages/keras/layers/convolutional.py", line 171, in call
    dilation_rate=self.dilation_rate)
  File "/Users/zacharyfrederick/opt/anaconda3/lib/python3.7/site-packages/keras/backend/tensorflow_backend.py", line 3650, in conv2d
    data_format=tf_data_format)
  File "/Users/zacharyfrederick/opt/anaconda3/lib/python3.7/site-packages/tensorflow/python/ops/nn_ops.py", line 851, in convolution
    return op(input, filter)
  File "/Users/zacharyfrederick/opt/anaconda3/lib/python3.7/site-packages/tensorflow/python/ops/nn_ops.py", line 966, in __call__
    return self.conv_op(inp, filter)
  File "/Users/zacharyfrederick/opt/anaconda3/lib/python3.7/site-packages/tensorflow/python/ops/nn_ops.py", line 591, in __call__
    return self.call(inp, filter)
  File "/Users/zacharyfrederick/opt/anaconda3/lib/python3.7/site-packages/tensorflow/python/ops/nn_ops.py", line 208, in __call__
    name=self.name)
  File "/Users/zacharyfrederick/opt/anaconda3/lib/python3.7/site-packages/tensorflow/python/ops/gen_nn_ops.py", line 1026, in conv2d
    data_format=data_format, dilations=dilations, name=name)
  File "/Users/zacharyfrederick/opt/anaconda3/lib/python3.7/site-packages/tensorflow/python/framework/op_def_library.py", line 788, in _apply_op_helper
    op_def=op_def)
  File "/Users/zacharyfrederick/opt/anaconda3/lib/python3.7/site-packages/tensorflow/python/util/deprecation.py", line 507, in new_func
    return func(*args, **kwargs)
  File "/Users/zacharyfrederick/opt/anaconda3/lib/python3.7/site-packages/tensorflow/python/framework/ops.py", line 3300, in create_op
    op_def=op_def)
  File "/Users/zacharyfrederick/opt/anaconda3/lib/python3.7/site-packages/tensorflow/python/framework/ops.py", line 1823, in __init__
    control_input_ops)
  File "/Users/zacharyfrederick/opt/anaconda3/lib/python3.7/site-packages/tensorflow/python/framework/ops.py", line 1662, in _create_c_op
    raise ValueError(str(e))
ValueError: Negative dimension size caused by subtracting 3 from 2 for 'conv2d_3/convolution' (op: 'Conv2D') with input shapes: [?,15,2,64], [3,3,64,64].

Answer 1

You have this error because your Kernels and strides are too big for your inputs, a common start is to use kernels of shape (3, 3) and strides (1, 1) .

Try reading on how a convolution is computed to give you intuition on how to set the correct kernel/stride size: http://cs231n.github.io/convolutional.networks/

Moreover you have an input with channel first , so you setted your first conv with channel first , it's great but you to do this for all your convolutions, because by default keras convolution's will use channel last .

For exemple, this is working:

input_shape = (4, 30, 180)
model = Sequential()
model.add(Conv2D(32, (8, 8), strides=(4, 4), activation='relu', input_shape=(4, 30, 180), data_format='channels_first'))
model.add(Activation('relu'))
model.add(Conv2D(64, (4, 4), strides=(1, 1), data_format='channels_first'))
model.add(Activation('relu'))
model.add(Conv2D(64, (3, 3), strides=(1, 1), data_format='channels_first'))
model.add(Activation('relu'))
model.add(Flatten())
model.add(Dense(512))
model.add(Activation('relu'))
model.add(Dense(2))
model.add(Activation('linear'))

Answer 2

The other answer is correct in diagnosis: after convolutions your image gets reduced and at some point the kernel becomes bigger than image. Try

1) lowering your kernel sizes or

2) adding , padding='same' to your convolution layer.

use Calculate the Output size in Convolution layer to calculate your outputs sizes.