How to input several time-series to LSTM in keras

Question

I have about 1000 nodes dataset where each node has 4 time-series. Each time series is exactly 6 length long.The label is 0 or 1 (ie binary classification).

More precisely my dataset looks as follows.

node, time-series1, time_series2, time_series_3, time_series4, Label
n1, [1.2, 2.5, 3.7, 4.2, 5.6, 8.8], [6.2, 5.5, 4.7, 3.2, 2.6, 1.8], …, 1
n2, [5.2, 4.5, 3.7, 2.2, 1.6, 0.8], [8.2, 7.5, 6.7, 5.2, 4.6, 1.8], …, 0
and so on.

Since, I have time-series, I assumed that it is more like sequence classification where most of the blog posts have used LSTM. I have never worked with LSTMs before and this is going to be my first ever keras application. So, I started working with the most basic sequential LSTM template as follows.

# create the model
model = Sequential()
model.add(Embedding(5000, 32, input_length=24))
model.add(LSTM(100))
model.add(Dense(1, activation='sigmoid'))
model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])
print(model.summary())

However, I am having difficulties in understanding how I can input four time-series to LSTM using the embedding layer. Can I use four embedding layers or are there any other way to resolve this issue?

I am happy to provide more details if needed.

Answer 1

You can treat each sequence within each node as a separate channel. So, first of all reshape your data into n_samples (I think you are calling this nodes), n_steps, and n_channel. With the data set you have, n_steps=6 and n_channel=4 . Basically you are stacking your different time series data per each sample (node).

Then, you can create you LSTM model. Something like this:

model = Sequential()
model.add(LSTM(64, input_shape=(6, 4), activation='sigmoid'))
model.add(Dense(1))
model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])
print(model.summary())

Note that you can have different activations with LSTM and may not necessarily need a Dense layer. But a Dense layer is an additional nonlinear layer and might improve the performance.

You can also stack LSTM layers to get more complex models like this:

model = Sequential()
model.add(LSTM(64, input_shape=(6, 4), return_sequences=True))
model.add(LSTM(1, input_shape=(6, 4)))
model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])
print(model.summary())

Answer 2

There are several ways you could structure this. For starters I would probably just flatten the 4 series into an array of 24 numbers and use Dense layers

from keras.models import Sequential
from keras.layers import Dense

model = Sequential()
model.add(Dense(32, input_shape=24))
model.add(Dense(32))
model.add(Dense(1, activation='sigmoid'))
model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])

print(data.shape) # (1000, 24)
model.fit(data, target)

You could have 4 separate inputs that process 4 series individually with LSTM layers. But you'd need to use the functional API. eg

input1 = Input(shape=(6,1))
x1 = LSTM(10)(input1)
input2 = Input(shape=(6,1))
x2 = LSTM(10)(input2)
input3 = Input(shape=(6,1))
x3 = LSTM(10)(input3)
input4 = Input(shape=(6,1))
x4 = LSTM(10)(input4)

x = concatenate([x1,x2,x3,x4])
x = Drouput(0.2)(x)
x = Dense(40)(x)
x = Drouput(0.2)(x)
output = Dense(1, activation='sigmoid')(x)

model = Model(inputs=[input1,input2,input3,input4], outputs=output)
model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])

print(data1.shape) #(1000,6,1)
model.fit([data1,data2,data3,data4], target)

Or if your data is in the shape (1000, 6, 4) you could use one LSTM and treat each series as a separate feature

model = Sequential()
model.add(LSTM(10, input_shape=(6,4)))
model.add(Dense(32))
model.add(Dense(1, activation='sigmoid'))
model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])

print(data.shape) # (1000, 6, 4)
model.fit(data, target)

Or you could use a CNN instead of an RNN.

model = Sequential()
model.add(Conv1D(10, kernel_size=3, input_shape=(6,4)))
model.add(Conv1D(10, kernel_size=2))
model.add(GlobalMaxPooling1D())
model.add(Dense(10))
model.add(Dense(1, activation='sigmoid'))
model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])

print(data.shape) # (1000, 6, 4)
model.fit(data, target)