TensorFlow2 / Keras：当子类化 keras.Model 时 input_shape 似乎没有效果

Question

When accessing the output of an intermediate layer I always received the error message: AttributeError: Layer l has no inbound nodes. I read that one has to determine the input_shape in the entry layer to overcome this problem:

class MyModel(tf.keras.Model):
    def __init__(self):
        super(MyModel, self).__init__()
        self.flatten = tf.keras.layers.Flatten(input_shape=(28, 28))
        self.dense = tf.keras.layers.Dense(64, activation=tf.nn.relu)
        self.classifier = tf.keras.layers.Dense(10, name='classifier')

    def call(self, inputs):
        x = self.flatten(inputs)
        x = self.dense(x)
        return self.classifier(x)

Unfortunately, this didn't work for me.

After this failed attempt, I tried to reconstruct my model using tf.keras.Sequential and again specified the input_shape for the first layer:

tf.keras.Sequential([
    tf.keras.layers.Flatten(input_shape=(28, 28)),
    tf.keras.layers.Dense(64, activation=tf.nn.relu),
    tf.keras.layers.Dense(10, name='classifier')
])

This, on the other hand worked!

So I asked myself why the first approach didn't work. To test this I specified arbitrary/wrong input_shapes for the subclassed model like this:

class WrongInputShapeModel(tf.keras.Model):
    def __init__(self):
        super(WrongInputShapeModel, self).__init__()
        self.flatten = tf.keras.layers.Flatten(input_shape=(42, 42, 42))
        # ...

I noticed that this model still functioned with MNIST (which has 28x28 images). That let me to believe that the keyword input_shape has no effect when defining a model subclassing tf.keras.Model .

Is this a bug, or am I missing something?

Answer 1

From your model subclassing code, it looks like you are not returning the model's forward-pass output. Maybe that's the reason it didn't work.

This should work:

import tensorflow as tf

class MyModel(tf.keras.Model):
    def __init__(self):
        super().__init__()
        self.flatten = tf.keras.layers.Flatten()
        self.dense = tf.keras.layers.Dense(64, activation=tf.nn.relu)
        self.classifier = tf.keras.layers.Dense(10, name='classifier', activation='softmax')

    def call(self, inputs):
        x = self.flatten(inputs)
        x = self.dense(x)
        return self.classifier(x)

And then, just as an example:

(x_train, y_train), (x_test, y_test) = tf.keras.datasets.mnist.load_data()

model = MyModel()
model.compile(optimizer=tf.keras.optimizers.Adam(1e-3), loss=tf.keras.losses.SparseCategoricalCrossentropy(),
             metrics=[tf.keras.metrics.SparseCategoricalAccuracy()])
history = model.fit(x_train, y_train, epochs=5)

Now, Flatten input shape is actually defined by the shape of the layer's input tensor, so it does not require an input_shape argument. input_shape is an attribute of Layer , so you can pass it, but the layer's forward-pass doesn't use it.