[英]Keras Transfer-Learning setting layers.trainable to True has no effect
I want to f.netune efficien.net using tf.keras (tensorflow 2.3) but i cannot change the training status of layers properly.我想使用 tf.keras (tensorflow 2.3) f.netune efficien.net 但我无法正确更改图层的训练状态。 My model looks like this:
我的 model 看起来像这样:
data_augmentation_layers = tf.keras.Sequential([
keras.layers.experimental.preprocessing.RandomFlip("horizontal_and_vertical"),
keras.layers.experimental.preprocessing.RandomRotation(0.8)])
efficientnet = EfficientNetB3(weights="imagenet", include_top=False,
input_shape=(*img_size, 3))
#Setting to not trainable as described in the standard keras FAQ
efficientnet.trainable = False
inputs = keras.layers.Input(shape=(*img_size, 3))
augmented = augmentation_layers(inputs)
base = efficientnet(augmented, training=False)
pooling = keras.layers.GlobalAveragePooling2D()(base)
outputs = keras.layers.Dense(5, activation="softmax")(pooling)
model = keras.Model(inputs=inputs, outputs=outputs)
model.compile(loss="categorical_crossentropy", optimizer=keras_opt, metrics=["categorical_accuracy"])
This is done so that my random weights on the custom top wont destroy the weights asap.这样做是为了让我在自定义顶部的随机重量不会尽快破坏重量。
Model: "functional_1"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
input_2 (InputLayer) [(None, 512, 512, 3)] 0
_________________________________________________________________
sequential (Sequential) (None, 512, 512, 3) 0
_________________________________________________________________
efficientnetb3 (Functional) (None, 16, 16, 1536) 10783535
_________________________________________________________________
global_average_pooling2d (Gl (None, 1536) 0
_________________________________________________________________
dense (Dense) (None, 5) 7685
=================================================================
Total params: 10,791,220
Trainable params: 7,685
Non-trainable params: 10,783,535
Everything seems to work until this point.到目前为止,一切似乎都有效。 I train my model for 2 epochs and then i want to start fine-tuning the efficien.net base.
我训练我的 model 2 个时期,然后我想开始微调 efficien.net 基础。 Thus i call
因此我打电话
for l in model.get_layer("efficientnetb3").layers:
if not isinstance(l, keras.layers.BatchNormalization):
l.trainable = True
model.compile(loss="categorical_crossentropy", optimizer=keras_opt, metrics=["categorical_accuracy"])
I recompiled and print the summary again to see that the number of non-trainable weights remained the same.我重新编译并再次打印摘要,发现不可训练权重的数量保持不变。 Also fitting does not bring better results that keeping frozen.
此外,贴合不会带来比保持冷冻更好的效果。
dense (Dense) (None, 5) 7685
=================================================================
Total params: 10,791,220
Trainable params: 7,685
Non-trainable params: 10,783,535
Ps: I also tried efficien.net3.trainable = True
but this also had no effect. Ps:我也试过
efficien.net3.trainable = True
但这也没有效果。
Could it be that it has something to do with the fact that i'm using a sequential and a functional model at the same time?难道这与我同时使用顺序和功能 model 这一事实有关吗?
For me the problem was using sequential API for part of the model. When I change to sequential, my model.sumary() displayed all the sublayers and it was possible to set some of them as trainable and others not.对我来说,问题是对 model 的一部分使用顺序 API。当我更改为顺序时,我的 model.sumary() 显示了所有子层,并且可以将其中一些设置为可训练的,而另一些则不能。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.