[英]Retraining the last layer of Inception-ResNet-v2
I am trying to retrain the last layer of inception-resnet-v2. 我正在尝试重新训练最后一层inception-resnet-v2。 Here's what I came up with: 这是我想出的:
train_op
to minimise only these variables wrt loss 创建一个train_op
以尽量减少这些变量损失 And I implemented that as follows: 我实现如下:
with slim.arg_scope(arg_scope):
logits = model(images_ph, is_training=True, reuse=None)
loss = tf.reduce_mean(tf.nn.sparse_softmax_cross_entropy_with_logits(logits, labels_ph))
accuracy = tf.contrib.metrics.accuracy(tf.argmax(logits, 1), labels_ph)
train_list = tf.get_collection(tf.GraphKeys.TRAINABLE_VARIABLES, 'InceptionResnetV2/Logits')
optimizer = tf.train.AdamOptimizer(learning_rate=FLAGS.learning_rate)
train_op = optimizer.minimize(loss, var_list=train_list)
# restore all variables whose names doesn't contain 'logits'
restore_list = tf.get_collection(tf.GraphKeys.TRAINABLE_VARIABLES, scope='^((?!Logits).)*$')
saver = tf.train.Saver(restore_list, write_version=tf.train.SaverDef.V2)
with tf.Session() as session:
init_op = tf.group(tf.local_variables_initializer(), tf.global_variables_initializer())
session.run(init_op)
saver.restore(session, '../models/inception_resnet_v2_2016_08_30.ckpt')
# followed by code for running train_op
This doesn't seem to work (training loss, error don't improve much from initial values). 这似乎不起作用(训练损失,错误从初始值不会有太大改善)。 Is there a better/elegant way to do this? 有没有更好/更优雅的方式来做到这一点? It would be good learning for me if you can also tell me what's going wrong here. 如果你也可以告诉我这里出了什么问题,那对我来说会很好。
There are several things: 有几件事:
is the scope correctly set? 范围是否正确设置? if you don't use L2 regularization and batch normalization of the gradients you might fall into a local minimum very soon and the network is unable to learn 如果您不使用L2正则化和梯度的批量标准化,您可能很快就会陷入局部最小值并且网络无法学习
from nets import inception_resnet_v2 as net with net.inception_resnet_v2_arg_scope(): logits, end_points = net.inception_resnet_v2(images_ph, num_classes=num_classes, is_training=True)
you should add the regularization variables to the loss (or at least the ones of the last layer): 你应该将正则化变量添加到损失(或至少是最后一层):
regularization_losses = tf.get_collection(tf.GraphKeys.REGULARIZATION_LOSSES) all_losses = [loss] + regularization_losses total_loss = tf.add_n(all_losses, name='total_loss')
training only the full connected layer might not be a good idea, I would train all the network as the features you need for your class aren't necessarily defined in the last layer but few layers before and you need to change them. 训练只有完整的连接层可能不是一个好主意,我会训练所有的网络,因为您的类所需的功能不一定在最后一层定义,但之前几层,你需要更改它们。
double check the train_op runs after the loss: 在损失后仔细检查train_op运行:
with ops.name_scope('train_op'): train_op = control_flow_ops.with_dependencies([train_op], total_loss)
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.