How to derive the output from Keras model using its layers' weight and biases?

Question

My model is an attempt to predict the values in a linear regression following y = 2x + 5. Therefore, my training data resembles the following:

x_train = [0, 1, 2, 3, 4, ...] and y_train = [5, 7, 9, 11, 13, ...]   

My Keras model looks like this:

`model = tf.keras.Sequential([
    tf.keras.layers.Dense(16, activation='relu', input_dim=1),
    tf.keras.layers.Dense(1, activation='linear')
])
model.compile(loss='mean_squared_error', optimizer='adam', metrics=['mse'])
model.fit(x_train, y_train, epochs=1000, batch_size=10)`

After getting a good validation accuracy, I have the desire to look at my model's weights and biases to calculate the output given an input because I wish to understand how a simple neural network works. I achieve the following weights and biases when running

for l in model.layers: print(l.get_weights())

[array([[-0.10382611,  0.48899287, -0.36912352, -0.11604425,  0.03658517,
     0.546786  , -0.0094437 ,  0.5393126 , -0.36325318, -0.20389882,
    -0.00112574, -0.39811927, -0.25433052, -0.16315842,  0.6172162 ,
    -0.47300738]], dtype=float32), array([ 0.        ,  1.1705374 ,  
     0.        ,  0.        , -0.41323203, 0.97515434, 0.        , 
     0.99699414,  0.        ,  0.        ,-0.2316811 , 0.        ,
     0.        ,  0.        ,  1.4638424 , 0.       ], dtype=float32)]
[array([[-0.30404267],
   [ 0.91265625],
   [ 0.3578334 ],
   [-0.23462006],
   [-0.33843294],
   [ 1.080244  ],
   [-0.5933689 ],
   [ 1.0348322 ],
   [ 0.47716653],
   [ 0.18852347],
   [-0.21219982],
   [ 0.45529807],
   [ 0.39576346],
   [-0.05013525],
   [ 0.67550814],
   [-0.19761673]], dtype=float32), array([0.7426254], dtype=float32)]

I am under the impression that if I were to throw in a value of 10 I should expect a value of 25 as the output (or very close). However, when I try and do the math myself, I am not so close. My current understanding of how this should work is:

1. Multiply the nth element of the weights array by 10 and add the nth element of the bias array
2. Take the nth result and multiply by the nth element of the second weights array and add the nth element of the second bias array
3. Result should be 25 (or very close)

Am I not understanding how this should work?

Answer 1

If we observe the weights and bias array of each layer we can see that the wrong result is due to lack of training. Model is not able to learn the pattern with these parameters, but we can say that it is improving as weights and bias are seen to be increasing which is required as we already have logic behind the pattern. I would suggest you to increase number of iterations and change batch size if your training data is small.