I am writing neural-network for predicate value in the time series

Question

dear friends!

At this moment I am trying to write code of Neural-network in keras, which will predict some value in time series. The time series have form like "0...0 N 0...0 N 0...0", where number of zeros between N's is the same.

For this target i am using LSTM-layers. I've been struggling with this task for over a week, but my network is really bad now

For this target i am using LSTM-layers. I've been struggling with this task for over a week, but my network is really bad yet (loss are very big and they aren't reduced during fit)

Mo model looks like

model =  Sequential()

model.add(LSTM(60, activation = softplus, use_bias = True, return_sequences = True, input_shape = (1, sample)))

model.add(LSTM(60, activation = softplus, use_bias = True, return_sequences = True))

model.add(LSTM(60, activation = softplus, use_bias = True))

model.add(Dropout(0.05))

model.add(Dense(1))

model.compile(loss = 'MSE',
              optimizer = 'RMSProp',
              metrics = ['accuracy', 'mae'])
model.fit(
        x = trainX, y = trainY, 
        batch_size = batch_size, 
        epochs = 1000,
        shuffle=True,
        validation_data=(testX, testY),
        callbacks = [cp_callback])

What wrong with this code? And what should I do to make my network better?

Thank you for answer!

Ps: I am really new in Neural Networks, so I am really sorry if my question is stupid. And sorry for my English too:)

Answer 1

You might want to try simplifying the network, also scaling the input down from 1000 to 1 might help. I created a simplified network which has good accuracy which might help point you in the right direction. I hope this helps.

from keras.models import Sequential
from keras.layers import LSTM, Dense
import numpy as np

data_dim = 1
timesteps = 11

# expected input data shape: (batch_size, timesteps, data_dim)
model = Sequential()
model.add(LSTM(4, input_shape=(timesteps, data_dim)))  # returns a sequence of vectors of dimension 4
model.add(Dense(2))
model.add(Dense(1))

model.compile(loss='mse',
              optimizer='rmsprop',
              metrics=['accuracy'])

# Generate dummy training data

x_train = np.array([[0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0],
                    [1, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0]])
x_train = np.expand_dims(x_train,axis=2)
y_train = np.array([[1], [0]])

model.summary()

print(x_train.shape)
print(y_train.shape)

model.fit(x_train, y_train, batch_size=2, epochs=200)