Dimension of shape in conv1D

Question

I have tried to build a CNN with one layer, but I have some problem with it. Indeed, the compilator says me that

ValueError: Error when checking model input: expected conv1d_1_input to have 3 dimensions, but got array with shape (569, 30)

This is the code

import numpy
from keras.models import Sequential
from keras.layers.convolutional import Conv1D
numpy.random.seed(7)
datasetTraining = numpy.loadtxt("CancerAdapter.csv",delimiter=",")
X = datasetTraining[:,1:31]
Y = datasetTraining[:,0]
datasetTesting = numpy.loadtxt("CancereEvaluation.csv",delimiter=",")
X_test = datasetTraining[:,1:31]
Y_test = datasetTraining[:,0]
model = Sequential()
model.add(Conv1D(2,2,activation='relu',input_shape=X.shape))
model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])
model.fit(X, Y, epochs=150, batch_size=5)
scores = model.evaluate(X_test, Y_test)
print("\n%s: %.2f%%" % (model.metrics_names[1], scores[1]*100))

Answer 1

td; lr you need to reshape you data to have a spatial dimension for Conv1d to make sense:

X = np.expand_dims(X, axis=2) # reshape (569, 30) to (569, 30, 1) 
# now input can be set as 
model.add(Conv1D(2,2,activation='relu',input_shape=(30, 1))

Essentially reshaping a dataset that looks like this:

features    
.8, .1, .3  
.2, .4, .6  
.7, .2, .1  

To:

[[.8
.1
.3],

[.2,
 .4,
 .6
 ],

[.3,
 .6
 .1]]

Explanation and examples

Normally convolution works over spatial dimensions. Kernel is "convolved" over the dimension producing a tensor. In the case of Conv1D, the kernel is passed of over the 'steps' dimension of every example.

You will see Conv1D used in NLP where steps is number of words in the sentence (padded to some fixed maximum length). The words would might be encoded as vectors of length 4.

Here is an example sentence:

jack   .1   .3   -.52   |
is     .05  .8,  -.7    |<--- kernel is `convolving` along this dimension.
a      .5   .31  -.2    |
boy    .5   .8   -.4   \|/

And the way we would set the input to the conv in this case:

maxlen = 4
input_dim = 3
model.add(Conv1D(2,2,activation='relu',input_shape=(maxlen, input_dim))

In your case you will treat the features as spatial dimension with each feature having length 1. (see below)

Here would be an example from your dataset

att1   .04    |
att2   .05    |  < -- kernel convolving along this dimension
att3   .1     |       notice the features have length 1. each
att4   .5    \|/      example have these 4 featues.

And we would set the Conv1D example as:

maxlen = num_features = 4 # this would be 30 in your case
input_dim = 1 # since this is the length of _each_ feature (as shown above)

model.add(Conv1D(2,2,activation='relu',input_shape=(maxlen, input_dim))

As you see your dataset has to be reshaped in to (569, 30, 1) use:

X = np.expand_dims(X, axis=2) # reshape (569, 30, 1) 
# now input can be set as 
model.add(Conv1D(2,2,activation='relu',input_shape=(30, 1))

Here is a full-fledged example that you can run (I'll use the Functional API )

from keras.models import Model
from keras.layers import Conv1D, Dense, MaxPool1D, Flatten, Input
import numpy as np

inp =  Input(shape=(5, 1))
conv = Conv1D(filters=2, kernel_size=2)(inp)
pool = MaxPool1D(pool_size=2)(conv)
flat = Flatten()(pool)
dense = Dense(1)(flat)
model = Model(inp, dense)
model.compile(loss='mse', optimizer='adam')

print(model.summary())

# get some data
X = np.expand_dims(np.random.randn(10, 5), axis=2)
y = np.random.randn(10, 1)

# fit model
model.fit(X, y)

Answer 2

I have mentioned this in other posts also:

To input a usual feature table data of shape (nrows, ncols) to Conv1d of Keras, following 2 steps are needed:

xtrain.reshape(nrows, ncols, 1)
# For conv1d statement: 
input_shape = (ncols, 1)

For example, taking first 4 features of iris dataset:

To see usual format and its shape:

iris_array = np.array(irisdf.iloc[:,:4].values)
print(iris_array[:5])
print(iris_array.shape)

The output shows usual format and its shape:

[[5.1 3.5 1.4 0.2]
 [4.9 3.  1.4 0.2]
 [4.7 3.2 1.3 0.2]
 [4.6 3.1 1.5 0.2]
 [5.  3.6 1.4 0.2]]

(150, 4)

Following code alters the format:

nrows, ncols = iris_array.shape
iris_array = iris_array.reshape(nrows, ncols, 1)
print(iris_array[:5])
print(iris_array.shape)

Output of above code data format and its shape:

[[[5.1]
  [3.5]
  [1.4]
  [0.2]]

 [[4.9]
  [3. ]
  [1.4]
  [0.2]]

 [[4.7]
  [3.2]
  [1.3]
  [0.2]]

 [[4.6]
  [3.1]
  [1.5]
  [0.2]]

 [[5. ]
  [3.6]
  [1.4]
  [0.2]]]

(150, 4, 1)

This works well for Conv1d of Keras. For input_shape (4,1) is needed.

Answer 3

I had a sparse matrix as input, so I couldn't reshape it without casting to usual array

The solution was to use the keras Reshape layer:

from keras.layers.core import Reshape

...
model = Sequential()
model.add(Reshape((X.shape[1], 1), input_shape=(X.shape[1], )))
model.add(Conv1D(2,2,activation='relu'))
...

Answer 4

Without being able to see more detail your data is not in the right shape after preprocessing.
Reshape X to have 3 dimensions:

np.reshape(X, (1, X.shape[0], X.shape[1]))

Answer 5

For sparse matrix, in my case, rows: 73196, column : 101 I used reshape function of numpy after converting my sparse matrix to array by array_ = sparse_matrix.A then I used below code

x_train_all = np.reshape(array_ , (73196, 101,1))

and in the input layer I used below code:

input2 = Input(shape=(101,1), dtype='float32', name='input2')