Tensorflow 2：为什么多层感知器神经网络中的`W`s和`b`s有相同的`name`？

Question

I'm new to TensorFlow 2 and reading the docs: https://www.tensorflow.org/api_docs/python/tf/Module

On this page, the part related to my question is: MLP (copy-paste from there):

class MLP(tf.Module):
  def __init__(self, input_size, sizes, name=None):
    super(MLP, self).__init__(name=name)
    self.layers = []
    with self.name_scope:
      for size in sizes:
        self.layers.append(Dense(input_dim=input_size, output_size=size))
        input_size = size
  @tf.Module.with_name_scope
  def __call__(self, x):
    for layer in self.layers:
      x = layer(x)
    return x

and I don't understand why the output of the following:

>>> module = MLP(input_size=5, sizes=[5, 5])
>>> module.variables
(<tf.Variable 'mlp/b:0' shape=(5,) ...>,
<tf.Variable 'mlp/w:0' shape=(5, 5) ...>,
<tf.Variable 'mlp/b:0' shape=(5,) ...>,
<tf.Variable 'mlp/w:0' shape=(5, 5) ...>,
)

Where I expect mlp/b:1 and mlp/w:1 would appear. I also tried the same code on my machine and got the same result on name , ie both mlp/b:0 and mlp/w:0 appear twice. Can anyone help me point out which point I have missed? Would the result mean that the same W , b are reused?

Answer 1

From the docs,

A tf.Variable represents a tensor whose value can be changed by running ops on it. Specific ops allow you to read and modify the values of this tensor. Higher level libraries like tf.keras use tf.Variable to store model parameters.

The :0 is not in any way the layer number. It is used to represent output tensor of an op in the underlying API.

For example, tf.Variable allocates one tensor [:0] , whereas 3-way split via tf.split allocates three tensors [:0,:1,:2] for their respective op in computational graph

tf.Variable([1])
# has output
# <tf.Variable 'Variable:0' shape=(1,) dtype=int32, numpy=array([1], dtype=int32)>

and

tf.compat.v1.disable_eager_execution()
a,b,c = tf.split([1,1,1], 3)
print(a.name)   # split:0
print(b.name)   # split:1
print(c.name)   # split:3

Refer to this post