Neural network Prediction becomes a straight line

Question

I implemented a two-layer neural network (according to the Kolmogorov-Arnold theorem, this is enough to represent any nonlinear function of n variables) to predict time series. However, by the end of the neural network, the volatility of the received forecast drops to almost zero and it turns into a straight line (I attach the forecast screen and the source code of the neural network). I increased the number of neurons in the hidden layer, the number of epochs, the size of the training sample, the learning rate, changed the range of normalization of the training sample data, changed the range of initial weights. Nothing helps. The size of the training sample is 336 examples, the training method is the reverse propagation of the error, the normalization method is minimax. Moreover, when using the hyperbolic tangent as an activation function, the situation improves somewhat, but the graph also looks strange. A "direct forecast" is output from ReLU. Does anyone have any ideas on this problem?

import random
import sys
import numpy
import math

eta=0.0001 #learning rate
n=200 #number of training epoch. There were also 500, 1000, 5000
inp=30 #input layer size
m=60 #hidden layer size
y=0 #output signal
t=0 #target signal
e=0 #error
d_y=0 #local gradient for the last neuron
err=0 #calculated network error for output neuron
err_av=0 #average network error
path='dataTrain.txt' #training sample
path2='dataLaunch.txt' #launching a forecast
day = 365 #number of days of forecasting
...

The rest is on the site: https://ideone.com/vV2QW6

Screenshot (activation function - sigmoid): https://ibb.co/GHrTGLr

Screenshot (activation function - hyperbolic tangent): https://ibb.co/WHFX3Sc

Thanks for attention.

Answer 1

This question is difficult to answer because you are trying to solve a time series problem using a tool that is not designed for that purpose. An RNN or LSTM would be more appropriate.

From what I understand reading the code you wrote, it seems you have fixed the input size (to 30 days) to predict the 31st day. Then you slide the 30 day window to include that prediction (and exclude the 1st day) so that you can predict the 32nd day. If my understanding is correct, the phenomenon you encountered can be explained (I hope) simply:

The prediction for day 31 and day 32 aren't expected to be that different because they are made from very similar inputs. Same for 32 and 33 etc. Eventually all the inputs become more similar (because these include the predictions which we've established are similar) and the outputs become more similar until you get something that looks constant.