negative dimension size on kears Conv1D

Question

I'm using Keras 's model api to apply a 1D convolution to an input 1d vector of size 20. I want five kernels of size 3 each. The input will be of shape (None, 1,20) (a variable number of 1D vectors of size 20).

input = Input(shape=(1, 20))
conv = Conv1D(filters=5, kernel_size=3, activation=keras.activations.relu, input_shape=(None,20, 1))(input)
dense =dense(1)(conv)
model = Model(inputs=input, outputs=dense)

model.compile(loss=nn.customLoss, optimizer='adam')

history = model.fit(train_X, train_labels, batch_size=50,
                    epochs=15, validation_split=0.2)

the summary of the model is -

_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
input_1 (InputLayer)         (None, None, 20)          0         
_________________________________________________________________
conv1d_1 (Conv1D)            (None, None, 5)           305       
_________________________________________________________________
dense_1 (Dense)              (None, None, 1)           6         
=================================================================
Total params: 311
Trainable params: 311
Non-trainable params: 0

train_x is of shape (None, 1, 20) , train_labels is of shape (None, 1) .

The error comes from the convolution layer -

    Caused by op 'conv1d_1/convolution/Conv2D', defined at:
  File "/home/user/Desktop/hack/imlhack2018/conv_nn.py", line 72, in <module>
    main()
  File "/home/user/Desktop/hack/imlhack2018/conv_nn.py", line 42, in main
    conv = Conv1D(filters=5, kernel_size=3, activation=keras.activations.relu, input_shape=(None,20, 1))(input)
  File "/home/user/anaconda3/lib/python3.6/site-packages/keras/engine/topology.py", line 596, in __call__
    output = self.call(inputs, **kwargs)
  File "/home/user/anaconda3/lib/python3.6/site-packages/keras/layers/convolutional.py", line 156, in call
    dilation_rate=self.dilation_rate[0])
  File "/home/user/anaconda3/lib/python3.6/site-packages/keras/backend/tensorflow_backend.py", line 3116, in conv1d
    data_format=tf_data_format)
  File "/home/user/anaconda3/lib/python3.6/site-packages/tensorflow/python/ops/nn_ops.py", line 670, in convolution
    op=op)
  File "/home/user/anaconda3/lib/python3.6/site-packages/tensorflow/python/ops/nn_ops.py", line 338, in with_space_to_batch
    return op(input, num_spatial_dims, padding)
  File "/home/user/anaconda3/lib/python3.6/site-packages/tensorflow/python/ops/nn_ops.py", line 662, in op
    name=name)
  File "/home/user/anaconda3/lib/python3.6/site-packages/tensorflow/python/ops/nn_ops.py", line 116, in _non_atrous_convolution
    name=scope)
  File "/home/user/anaconda3/lib/python3.6/site-packages/tensorflow/python/ops/nn_ops.py", line 2010, in conv1d
    data_format=data_format)
  File "/home/user/anaconda3/lib/python3.6/site-packages/tensorflow/python/ops/gen_nn_ops.py", line 399, in conv2d
    data_format=data_format, name=name)
  File "/home/user/anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py", line 767, in apply_op
    op_def=op_def)
  File "/home/user/anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 2506, in create_op
    original_op=self._default_original_op, op_def=op_def)
  File "/home/user/anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1269, in __init__
    self._traceback = _extract_stack()

InvalidArgumentError (see above for traceback): computed output size would be negative
     [[Node: conv1d_1/convolution/Conv2D = Conv2D[T=DT_FLOAT, data_format="NHWC", padding="VALID", strides=[1, 1, 1, 1], use_cudnn_on_gpu=true, _device="/job:localhost/replica:0/task:0/cpu:0"](conv1d_1/convolution/ExpandDims, conv1d_1/convolution/ExpandDims_1)]]

When I add padding="same" to the convolutional layer everything seems to be working just fine. What is the reason for this behavior?

Answer 1

Your input shape is (1, 20), which is interpreted as a 1 width, 20 channel array. You probably want the opposite, that is, width 20 and 1 channel. As your array has a single element, performing convolution without padding of SAME will lead to a negative dimension, which produces the error.

Note that convolution is always performed on the spatial dimensions, which for Conv1D the second to last dimension in the shape array. The last dimension represents the channels.

Answer 2

In the official documents，it writes “When using this layer as the first layer in a model, provide an input_shape argument (tuple of integers or None, does not include the batch axis)”. I am confused that why it declared the input shape of conv1D after it had in the Input layer