TensorFlow：是否可以为多GPU训练恢复检查点模型？

Question

I am currently using a supervisor and constructed just one graph to perform transfer learning using the pre-trained weights from TF-slim. I am wondering if there is a way to restore checkpoint models to multiple inference models at the outset? My primary concern is that firstly, the name scopes that are defined as in a reference code on the TF repository may cause the pre-trained variables to be unable to be restored due to a name mismatch. Also, given that I have to use a supervisor with an init_fn that takes in only one saver that restores the variables, how could I have multiple savers to restore the same variables to multiple GPUs (If I even need to have multiple savers at all).

One idea I have is that perhaps I could just restore the variables to one graph, and let the other GPUs use the same graph for training. However, would the training for the next GPU take place only after the first GPU has completed? But this way, I won't be able to restore the weights according to the original checkpoint model variable names too, unless I edit the names of the checkpoint weights.

Answer 1

关于保存和恢复变量的tensorflow文档将您指向saver对象，允许您在构造saver时通过将字典从保存的名称传递到变量对象来指定将哪些保存的变量恢复为模型变量。