Why do I get 6 parameters in Keras in simple 2 output 2 input network?

Question

I am learning about neural networks in keras. I specified a simple model on made up data.

model=tf.keras.models.Sequential()
model.add(tf.keras.layers.Dense(2, input_dim=2))
model.compile(optimizer='sgd', loss='mean_squared_error')

I have two attributes to predict two values.

Here is where I initialize my data:

x=[]
y=[]
for x1 in range (6):
    x2=int(random.random()*10)
    x.append([x1,x2])
    y.append([2*x1+x2**2-2, x1*x2])
xs = np.array(x, dtype=float)
xs=xs.reshape(6,2)
ys = np.array(y, dtype=float)
ys=ys.reshape(6,2)
model.fit(xs, ys, epochs=500)

Mind you, I use the data solely for the purpose of learning. After I attempted to observe the model. I run model.summary() and model.get_weights() .

Model: "sequential"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
dense (Dense)                (None, 2)                 6         
=================================================================
Total params: 6
Trainable params: 6
Non-trainable params: 0
_________________________________________________________________
None
model weights  [array([[0.5137405, 5.477211 ],
       [8.750836 , 1.6910588]], dtype=float32), array([-5.701193, -7.874653], dtype=float32)]

I don't understand why are there 6 params and six weights. From my understanding there should be two going from each input, or should I have somewhere specifically defined the output layer?

Answer 1

The model architecture you have defined is pictorially shown below

You have one dense layer with two neurons. Why two neurons? because the first parameter to Dense is units which denotes the number of neurons. Each neuron does linear operation of XW + b and then applies activation function over it. The learnable parameters in a nuerons are W and b .

Since the size of X is 2 (2 features) so size of W (=2) + b = 3. So each neuron in this case will have 3 parameters and 2 such will have 6 parameters.

Answer 2

You have a single output layer with two neurons, each of these neurons must have two weights (since the inputs are of dimension 2) and another weight called "bias". So each neuron has 3 weights.

In summary, you have 2 neurons and each one has 3 weights or trainable parameters, so in total there are 6 trainable parameters in your network.