Prediction by Neural network Keras is not working

Question

I want to write a NN program for recognition using Keras.

I have used 2 sets of data for training:

toyX = [1, 2, 3, 4, 5, 6, 7, 8]
toyX2 = [18, 17, 16, 15, 14, 13, 12, 11].

After training with toyX and then toyX2 , the output of model.predict(toyX) is [[0.56053144 1.0758346 1.7890009 ]] . However, it should have been [6, 11, 14] .

Should I add more layers or change parameters to improve prediction? Which parameters I should change?

Please help me to solve this problem.

from keras.models import Sequential
from keras.optimizers import Adam
from keras.layers import Conv1D, MaxPooling1D
from keras.layers import Dense, Flatten
from keras.layers import Dropout
import numpy as np

model = Sequential()
model.add(Conv1D(filters=64, kernel_size=3, activation='relu', input_shape=(8, 1)))
model.add(Conv1D(filters=64, kernel_size=3, activation='relu'))
model.add(Dropout(0.5))
model.add(MaxPooling1D(pool_size=2))
model.add(Flatten())
model.add(Dense(100, activation='relu'))
#model.add(Dense(3, activation='softmax'))
model.add(Dense(3))
#model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
model.compile(loss='mean_squared_error', optimizer='adam', metrics=['accuracy'])
print(model.summary())


toyX = np.array([1, 2, 3, 4, 5, 6, 7, 8]).reshape(1, 8, 1)
toyX2 = np.array([18, 17, 16, 15, 14, 13, 12, 11]).reshape(1, 8, 1)
#print (toyX.shape)
toyY = np.array([6, 11, 14]).reshape(1, 3)
#print (toyY.shape)
toyY2 = np.array([1, 2, 3]).reshape(1, 3) # if flatten is active

model.fit(toyX, toyY, epochs = 1000, verbose = 0)
model.fit(toyX2, toyY2, epochs = 1000, verbose = 0)

print (model.predict(toyX))

Answer 1

This phenomenon is called catastrophic forgetting of neural networks.

You can read a paper on this: https://arxiv.org/pdf/1708.02072.pdf

Your toyX and toyX2 has a completely different distribution. When you re-train your model with ToyX2 for 1000 epochs, your model has completely forgotten the mapping from toyX to toyY .

You should train only for very few epochs with a really small learning rate if you want to make sure the knowledge of previous training stays or just mix both of the sets and re-train.