Hi I'm new in keras and I concatenate two LSTM in keras. The dataset is a univariate time series which was split by the method sliding window. Then I reshaped it as a tesor of [samples, features,time steps]. However when I tried to fit the model the following error appears.
TypeError: in user code:
/usr/local/lib/python3.8/dist-packages/tensorflow/python/keras/engine/training.py:806 train_function *
return step_function(self, iterator)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/keras/engine/training.py:796 step_function **
outputs = model.distribute_strategy.run(run_step, args=(data,))
/usr/local/lib/python3.8/dist-packages/tensorflow/python/distribute/distribute_lib.py:1211 run
return self._extended.call_for_each_replica(fn, args=args, kwargs=kwargs)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/distribute/distribute_lib.py:2585 call_for_each_replica
return self._call_for_each_replica(fn, args, kwargs)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/distribute/distribute_lib.py:2945 _call_for_each_replica
return fn(*args, **kwargs)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/keras/engine/training.py:789 run_step **
outputs = model.train_step(data)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/keras/engine/training.py:747 train_step
y_pred = self(x, training=True)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/keras/engine/base_layer.py:985 __call__
outputs = call_fn(inputs, *args, **kwargs)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/keras/engine/sequential.py:386 call
outputs = layer(inputs, **kwargs)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/keras/engine/base_layer.py:982 __call__
self._maybe_build(inputs)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/keras/engine/base_layer.py:2643 _maybe_build
self.build(input_shapes) # pylint:disable=not-callable
/usr/local/lib/python3.8/dist-packages/tensorflow/python/keras/utils/tf_utils.py:323 wrapper
output_shape = fn(instance, input_shape)
/usr/local/lib/python3.8/dist-packages/tensorflow/python/keras/layers/merge.py:500 build
del reduced_inputs_shapes[i][self.axis]
TypeError: list indices must be integers or slices, not ListWrapper
How should I pass trainX and testX? I want to do somenthing like in this image:
The code is the following
trainX = numpy.reshape(trainX, (trainX.shape[0],1, trainX.shape[1]))
testX = numpy.reshape(testX, (testX.shape[0], 1,testX.shape[1]))
model1= Sequential()
model1.add(LSTM(6, input_shape=(1,look_back)))
model2 = Sequential()
model2.add(LSTM(6, input_shape=(1,look_back)))
model = Sequential()
model.add(Concatenate([model1, model1]))
model.add(Dense(1))
model.compile(loss='mean_squared_error', optimizer='adam', metrics = ['mse'])
model.fit([trainX, trainX], trainY, epochs=50, batch_size=1, verbose=1)
Try to write it like this:
...
model1= Sequential()
model1.add(LSTM(6, input_shape=(1,look_back)))
model2 = Sequential()
model2.add(LSTM(6, input_shape=(1,look_back)))
concatenated_models = concatenate([model1, model2])
out = Dense(1, activation='softmax', name='output')(concatenated_models)
added_model = Model([model1, model2], out)
added_model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])
added_model.fit([trainX, trainX], trainY, epochs=50, batch_size=1, verbose=1)
That might be working better. Not 100% sure (quite late here ;-))
You need to use Functional API in Keras .
Your model with Functional API:
from tensorflow.keras.layers import Dense, Concatenate, LSTM, Input, Flatten
from tensorflow.keras.models import Model
look_back = 3200 # just for running the code I used this number
# Model architecture
inputs = Input(shape=(1,look_back),name='Input_1')
lstm1 = LSTM(6, name='LSTM_1')(inputs)
lstm2 = LSTM(6, name='LSTM_2')(inputs)
concatenated = Concatenate( name='Concatenate_1')([lstm1,lstm2])
output1 = Dense(1, name='Dense_1')(concatenated)
model = Model(inputs=inputs, outputs=output1)
Now let's see the architecture:
model.summary()
output:
Model: "functional_13"
__________________________________________________________________________________________________
Layer (type) Output Shape Param # Connected to
==================================================================================================
Input_1 (InputLayer) [(None, 1, 3200)] 0
__________________________________________________________________________________________________
LSTM_1 (LSTM) (None, 6) 76968 Input_1[0][0]
__________________________________________________________________________________________________
LSTM_2 (LSTM) (None, 6) 76968 Input_1[0][0]
__________________________________________________________________________________________________
Concatenate_1 (Concatenate) (None, 12) 0 LSTM_1[0][0]
LSTM_2[0][0]
__________________________________________________________________________________________________
Dense_1 (Dense) (None, 1) 13 Concatenate_1[0][0]
==================================================================================================
Total params: 153,949
Trainable params: 153,949
Non-trainable params: 0
__________________________________________________________________________________________________
Also, we can take a better look at the layers and their connections by checking the model plot
The model plot:
from tensorflow.keras.utils import plot_model
plot_model(model)
output:
Now let's train the model:
I created some dummy data with sklearn for training the model and everything works fine.
Training the model:
from sklearn.datasets import make_blobs
train_x , train_y = make_blobs(n_samples=1000, centers=2, n_features=look_back,random_state=0)
train_x = train_x.reshape(train_x.shape[0], 1, train_x.shape[1])
model.compile(loss='mean_squared_error', optimizer='adam', metrics = ['mse'])
model.fit(train_x, train_y, epochs=5, batch_size=1, verbose=1)
output:
Epoch 1/5
1000/1000 [==============================] - 2s 2ms/step - loss: 0.0133 - mse: 0.0133
Epoch 2/5
1000/1000 [==============================] - 2s 2ms/step - loss: 1.4628e-13 - mse: 1.4628e-13
Epoch 3/5
1000/1000 [==============================] - 2s 2ms/step - loss: 2.2808e-14 - mse: 2.2808e-14
Epoch 4/5
1000/1000 [==============================] - 2s 2ms/step - loss: 5.2458e-15 - mse: 5.2458e-15
Epoch 5/5
1000/1000 [==============================] - 2s 2ms/step - loss: 1.1384e-15 - mse: 1.1384e-15
<tensorflow.python.keras.callbacks.History at 0x7f5fe4ce9f28>
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.