tf.variable_scope（）和tf.train.Saver（）有什么区别？

Question

I am building a neural network. I am using tf.variable_scope() for the train part and use the same scope for the test part (reset=True). I am wondering whether I will get different results if I use tf.train.Saver() to save the variables in the train part and use tf.train.restore() to restore the variables for the test part? Basically, I want to make sure the trained variables are copied to the test part.

For instance, in the example below, both variables v1 and v_1 have the same values, so, in this case, do I need to use tf.train.Saver() to get correct results? import tensorflow as tf tf.reset_default_graph()

# Create some variables.
with tf.variable_scope("first_model"):
    v1 = tf.get_variable("v1", shape=[3], initializer = tf.zeros_initializer)
    v2 = tf.get_variable("v2", shape=[5], initializer = tf.zeros_initializer)

    inc_v1 = v1.assign(v1+2)
    dec_v2 = v2.assign(v2-1)



with tf.variable_scope("first_model", reuse=True):
    # Create some variables.
    v_1 = tf.get_variable("v1", shape=[3])
    v_2 = tf.get_variable("v2", shape=[5])

    inc_v_1 = v1.assign(v1+2)
    dec_v_2 = v2.assign(v2-1)

 # Add an op to initialize the variables.
init_op = tf.global_variables_initializer()

# Add ops to save and restore all the variables.
saver = tf.train.Saver()

# Later, launch the model, initialize the variables, do some work, and save the
# variables to disk.
with tf.Session() as sess:
  sess.run(init_op)
  # Do some work with the model.
  inc_v1.op.run()
  dec_v2.op.run()

  inc_v_1.op.run()
  dec_v_2.op.run()
  # Save the variables to disk.
  save_path = saver.save(sess, "/tmp/model_2.ckpt")
  print("Model saved in path: %s" % save_path)


  print("v1 : %s" % v1.eval())
  print("v2 : %s" % v2.eval())

  print("v_1 : %s" % v_1.eval())
  print("v_2 : %s" % v_2.eval())

Answer 1

tf.variable_scope() changes your variable names.

For example, consider creating a variable with and without variable_scope :

x = tf.Variable(1.0, name='myvar')
<tf.Variable 'myvar:0' shape=() dtype=float32_ref>

Note the variable is named myvar:0 , when you save the checkpoint using tf.train.Saver() this will be the name of this variable. When you restore your checkpoint with tf.train.restore() you better have a variable named myvar:0 in your graph otherwise it won't know where to restore the variable to.

Now the same thing with tf.variable_scope() :

with tf.variable_scope('newscope'):
  x = tf.Variable(1.0, name='myvar')

<tf.Variable 'newscope/myvar:0' shape=() dtype=float32_ref>

Notice that the name is now newscope/myvar:0 . The name of the variable has changed. this allows you to keep your variable namespace organized. It's useful for debugging and for visualizing things in tensorboard mainly.

When you save and restore a model you have 2 approaches you can take.

The basic checkpoint just saves the data associated with the model (this appears to be what you've referenced in your question). In this paradigm, you need to re-create all of the variables in your graph before loading the checkpoint, and your names better all match (if you used variable_scope in training you better have done the same in test). This is the method I typically follow and I recommend you put all of your tensorflow ops in a function called build_graph() . Then to re-build the graph for your training or test, you just call that function and voila, the graph is identical and the save/restore functions work as expected.

You might also note that you can save the meta_graph and restore not only the variables, but also the graph definitions (the actual operations you're performing). In this paradigm you first load the meta_graph definitions, then the checkpoint, you don't need to re-define the graph in your test environment. I don't use this method much so I won't go into any more detail about it.