Error using CNN hyper parameter optimization for multi-target regression

Question

I am trying to solve a multi-target regression problem using CNN. To optimize hyper parameters, I have used Talos library, but I end up with an error:

Error when checking input: expected conv1d_1_input to have 3 dimensions, but got array with shape (280000, 70)

Description of Dataset: X has 70 features and Y has 32 targets Length of Time Series: 509760 time points

Shape of Dataset:

Train Data size: (400000, 70),
Train Target size: (400000, 32),
Test Data size: (109760, 70),
Test Target size: (109760, 32),

Code:

tf.keras.backend.clear_session()

def model(x_train, y_train, x_val, y_val, params):

    n_timesteps, n_features, n_outputs = x_train.shape[0],    x_train.shape[1], y_train.shape[1]

    model = Sequential()

    model.add(Conv1D(filters=params['conv1_filter'], kernel_size=(3), activation=params['activation'], input_shape=(n_timesteps,n_features)))
    model.add(Conv1D(filters=params['conv2_filter'], kernel_size=(3), activation=params['activation']))
    model.add(MaxPooling1D(pool_size=2))
    model.add(Dropout(params['dropout']))
    model.add(Conv1D(filters=8, kernel_size=3, activation=params['activation']))
    model.add(MaxPooling1D(pool_size=2))
    model.add(Dense(100, activation=params['activation']))
    model.add(Dense(n_outputs))

    model.add(Flatten())    
    model.compile(loss='mse',optimizer=params['optimizer'](),metrics=['acc', fmeasure_acc])


    out = model.fit(x_train, y_train, validation_data=[x_val, y_val], batch_size=params['batch_size'], epochs=params['epochs'],verbose=1,shuffle=True)

    return out, model

p = {'conv1_filter':[32],
     'conv2_filter':[32],
     'optimizer': [Nadam],
     'batch_size': [1000],
     'epochs': [5],
     'dropout': [0.5],
     'activation':[relu]}

# and run the experiment
t = ta.Scan(x=x_train,y=y_train,model=model,params=p,experiment_no='1')

The error is as follows:

  168 # input parameters section ends 169 --> 170 self._null = self.runtime() 171 172 def runtime(self): /anaconda3/lib/python3.7/site-packages/talos/scan/Scan.py in runtime(self) 173 174 self = scan_prepare(self) --> 175 self = scan_run(self) /anaconda3/lib/python3.7/site-packages/talos/scan/scan_run.py in scan_run(self) 16 disable=self.disable_progress_bar) 17 while len(self.param_log) != 0: ---> 18 self = scan_round(self) 19 self.pbar.update(1) 20 self.pbar.close() /anaconda3/lib/python3.7/site-packages/talos/scan/scan_round.py in scan_round(self) 30 # fit the model 31 try: ---> 32 _hr_out, self.keras_model = ingest_model(self) 33 except TypeError as err: 34 if err.args[0] == "unsupported operand type(s) for +: 'int' and 'numpy.str_'": /anaconda3/lib/python3.7/site-packages/talos/model/ingest_model.py in ingest_model(self) 8 self.x_val, 9 self.y_val, ---> 10 self.round_params) <ipython-input-9-b33b869928ed> in model(x_train, y_train, x_val, y_val, params) 20 21 ---> 22 out = model.fit(x_train, y_train, validation_data=[x_val, y_val], batch_size=params['batch_size'], epochs=params['epochs'],verbose=1,shuffle=True) 23 24 /anaconda3/lib/python3.7/site-packages/keras/engine/training.py in fit(self, x, y, batch_size, epochs, verbose, callbacks, validation_split, validation_data, shuffle, class_weight, sample_weight, initial_epoch, steps_per_epoch, validation_steps, **kwargs) 950 sample_weight=sample_weight, 951 class_weight=class_weight, --> 952 batch_size=batch_size) 953 # Prepare validation data. 954 do_validation = False /anaconda3/lib/python3.7/site-packages/keras/engine/training.py in _standardize_user_data(self, x, y, sample_weight, class_weight, check_array_lengths, batch_size) 749 feed_input_shapes, 750 check_batch_axis=False, # Don't enforce the batch size. --> 751 exception_prefix='input') 752 753 if y is not None: /anaconda3/lib/python3.7/site-packages/keras/engine/training_utils.py in standardize_input_data(data, names, shapes, check_batch_axis, exception_prefix) 126 ': expected ' + names[i] + ' to have ' + 127 str(len(shape)) + ' dimensions, but got array ' --> 128 'with shape ' + str(data_shape)) 129 if not check_batch_axis: 130 data_shape = data_shape[1:] ValueError: Error when checking input: expected conv1d_1_input to have 3 dimensions, but got array with shape (280000, 70) 

Answer 1

I looked at it again, a few of things needed to be changed in your code. I tested it and it should work now:

First of all, the model itself had a couple of issues.

Namely, the input to your model should be (n_features, 1) instead of (n_timesteps,n_features) . Keras internally thinks that you are ignoring the batch dimension when giving the input shape. So when what it will register as a shape is (batch_size, n_features, 1) . The last dimension is required so that it is compatible with Conv1D .

This is how it should look:

model.add(Conv1D(filters=params['conv1_filter'], kernel_size=(3), activation=params['activation'], input_shape=(n_features, 1))) 

Secondly, the model shouldn't end with a Flatten() layer. Instead this layer should go before the Dense() layer (so that it sees a 2D input instead of a 3D one). After both changes the model should look like this:

model = Sequential()

model.add(Conv1D(filters=params['conv1_filter'], kernel_size=(3), activation=params['activation'], input_shape=(n_features,1))) 
model.add(Conv1D(filters=params['conv2_filter'], kernel_size=(3), activation=params['activation']))
model.add(MaxPooling1D(pool_size=2))
model.add(Dropout(params['dropout']))
model.add(Conv1D(filters=8, kernel_size=3, activation=params['activation']))
model.add(MaxPooling1D(pool_size=2))
model.add(Flatten())
model.add(Dense(100, activation=params['activation']))
model.add(Dense(n_outputs))

Because we changed the input_shape of the model, we'll need to change the shape of our data. What needs to be done is to change (400000, 70) to (400000, 70, 1) and (109760, 70) to (109760, 70, 1) . This can be done with the following command:

x_train = np.expand_dims(x_train, axis=-1)
x_val = np.expand_dims(x_val, axis=-1)

I didn't test it on Talos, but the keras part works just fine.