[英]Are Keras custom layer parameters non-trainable by default?
I built a simple custom layer in Keras and was surprised to find that the parameters were not set to trainable by default.我在 Keras 中构建了一个简单的自定义层,惊讶地发现参数默认没有设置为可训练。 I can get it to work by explicitly setting the trainable attribute.
我可以通过显式设置可训练属性来使其工作。 I can't explain why this is by looking at documentation or code.
我无法通过查看文档或代码来解释为什么会这样。 Is this how it is supposed to be or I am doing something wrong which is making the parameters non-trainable by default?
这是它应该是这样,还是我做错了什么,默认情况下使参数不可训练? Code:
代码:
import tensorflow as tf
class MyDense(tf.keras.layers.Layer):
def __init__(self, **kwargs):
super(MyDense, self).__init__(kwargs)
self.dense = tf.keras.layers.Dense(2, tf.keras.activations.relu)
def call(self, inputs, training=None):
return self.dense(inputs)
inputs = tf.keras.Input(shape=10)
outputs = MyDense()(inputs)
model = tf.keras.Model(inputs=inputs, outputs=outputs, name='test')
model.compile(loss=tf.keras.losses.MeanSquaredError())
model.summary()
Output: Output:
Model: "test"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
input_1 (InputLayer) [(None, 10)] 0
_________________________________________________________________
my_dense (MyDense) (None, 2) 22
=================================================================
Total params: 22
Trainable params: 0
Non-trainable params: 22
_________________________________________________________________
If I change the custom layer creation like this:如果我像这样更改自定义图层创建:
outputs = MyDense(trainable=True)(inputs)
the output is what I expect (all parameters are trainable): output 是我所期望的(所有参数都是可训练的):
=================================================================
Total params: 22
Trainable params: 22
Non-trainable params: 0
_________________________________________________________________
then it works as expected and makes all the parameters trainable.然后它按预期工作并使所有参数可训练。 I don't understand why that is needed though.
我不明白为什么需要这样做。
No doubt, that's an interesting quirk.毫无疑问,这是一个有趣的怪癖。
When making a custom layer, a tf.Variable
will be automatically included in the list of trainable_variable
.制作自定义层时,
tf.Variable
将自动包含在trainable_variable
列表中。 You didn't use tf.Variable
, but a tf.keras.layers.Dense
object instead, which will not be treated as a tf.Variable
, and not set trainable=True
by default.您没有使用
tf.Variable
,而是使用tf.keras.layers.Dense
object ,它不会被视为tf.Variable
,并且默认情况下不设置 trainable trainable=True
。 However, the Dense
object you used will be set to trainable.但是,您使用的
Dense
object 将设置为可训练。 See:看:
MyDense().dense.trainable
True
If you used tf.Variable
(as it should), it will be trainable by default.如果您使用
tf.Variable
(应该如此),默认情况下它将是可训练的。
import tensorflow as tf
class MyDense(tf.keras.layers.Layer):
def __init__(self, units=2, input_dim=10):
super(MyDense, self).__init__()
w_init = tf.random_normal_initializer()
self.w = tf.Variable(
initial_value=w_init(shape=(input_dim, units), dtype="float32"),
trainable=True,
)
b_init = tf.zeros_initializer()
self.b = tf.Variable(
initial_value=b_init(shape=(units,), dtype="float32"), trainable=True
)
def call(self, inputs, **kwargs):
return tf.matmul(inputs, self.w) + self.b
inputs = tf.keras.Input(shape=10)
outputs = MyDense()(inputs)
model = tf.keras.Model(inputs=inputs, outputs=outputs, name='test')
model.compile(loss=tf.keras.losses.MeanSquaredError())
model.summary()
Model: "test"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
input_11 (InputLayer) [(None, 10)] 0
_________________________________________________________________
my_dense_18 (MyDense) (None, 2) 22
=================================================================
Total params: 22
Trainable params: 22
Non-trainable params: 0
_________________________________________________________________
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.