TensorFlow Object Detection API: training with `fine_tune_checkpoint` - unfreeze the earlier layers?

Question

I'm training models quite succesfully with fine_tune_checkpoint (proto link) from a pre-trained network from the model zoo.

However, as far as I know, with the finetuning variant of transfer learning, the earlier layers are frozen. Only the last layer(s) are actually updated during training.

Is this assumption correct, ie is it also implemented like this in the TF Object Detection API?

Assuming that it is, I'm wondering right now if I can squeeze out a bit more performance by training for some time without having the earlier layers being frozen. However, I cannot find a straight forward way to toggle between these two modes of training in the config file. Any ideas?

Answer 1

The default training configuration trains all weights. The fact that you're fine-tuning only means you load some/all of the weights rather than initializing them all (eg fine-tuning from ImageNet pre-trained model or fine-tuning from a pre-trained detection model). In order to freeze some weights, you need to state their names by pattern using the freeze_variables option in the train config. In order to specify variables that should be trained you can use the corresponding update_trainable_variables . The names can be found by inspecting the model/graph after building it.