Tensorflow-除非我重新创建“ saver”对象，否则无法初始化保存的变量。 为什么？

Question

I'm pretty sure I'm missing something about how tensorflow works because my solution doesn't make any sense.

I'm trying to train a neural network (from scratch, without using Estimators or other abstractions), save it, and load a simplified version of it for inference.

The following code trains but gives me the error: FailedPreconditionError (see above for traceback): Attempting to use uninitialized value hidden0/biases/Variable [[Node: hidden0/biases/Variable/read = Identity[T=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:CPU:0"](hidden0/biases/Variable)]] . If I add the commented line - if I recreate the saver obect that I'm not going to use nor return - the code works just fine.

Why do I need to create a (useless) saver object in order to restore the saved weights?

import tensorflow as tf
import numpy as np

def add_fc_layer(input_tensor, input_dimensions, output_dimensions, layer_name, activation=None):
    with tf.variable_scope(layer_name):
        with tf.variable_scope('weights'):
            weights = tf.Variable(tf.truncated_normal([input_dimensions, output_dimensions]))
        with tf.variable_scope('biases'):
            biases = tf.Variable(tf.zeros([output_dimensions]))
        with tf.variable_scope('Wx_plus_b'):
            preactivate = tf.matmul(input_tensor, weights) + biases
        if activation is None:
            return preactivate
        with tf.variable_scope('activation'):
            activations = activation(preactivate)
        return activations


def make_network(model_phase):
    if model_phase not in {"train", "test"}:
        raise ValueError("invalid type")

    hidden0_units = 25
    hidden1_units = 15
    hidden2_units = 10
    input_size = 10
    output_size = 4

    with tf.variable_scope('InputVector'):
        inputs = tf.placeholder(shape=[1, input_size], dtype=tf.float32)

    hidden0_out = add_fc_layer(inputs, input_size, hidden0_units, "hidden0", activation=tf.nn.sigmoid)
    hidden1_out = add_fc_layer(hidden0_out, hidden0_units, hidden1_units, "hidden1", activation=tf.nn.sigmoid)
    hidden2_out = add_fc_layer(hidden1_out, hidden1_units, hidden2_units, "hidden2", activation=tf.nn.sigmoid)

    out =  add_fc_layer(hidden2_out, hidden2_units, output_size, "regression")

    if model_phase == "test":
        # UNCOMMENTIN THIS LINE MAKES THE SCRIPT WORK
        # saver = tf.train.Saver(var_list=tf.trainable_variables())
        return inputs, out

    saver = tf.train.Saver(var_list=tf.trainable_variables())

    with tf.variable_scope('training'):
        with tf.variable_scope('groundTruth'):
            ground_truth = tf.placeholder(shape=[1, output_size], dtype=tf.float32)
        with tf.variable_scope('loss'):
            loss = tf.reduce_sum(tf.square(ground_truth - out))
            tf.summary.scalar('loss', loss)
        with tf.variable_scope('optimizer'):
            trainer = tf.train.AdamOptimizer(learning_rate=0.001)
        with tf.variable_scope('gradient'):
            updateModel = trainer.minimize(loss)


    with tf.variable_scope('predict'):
        predict = tf.random_shuffle(tf.boolean_mask(out, tf.equal(out, tf.reduce_max(out, axis=None))))[0]


    writer = tf.summary.FileWriter('/tmp/test', tf.get_default_graph())


    return inputs, out, ground_truth, updateModel, writer, saver


train_graph = tf.Graph()
with tf.Session(graph=train_graph) as sess:
    tf.set_random_seed(42)
    inputs, out, ground_truth, updateModel, writer, saver = make_network(model_phase='train')
    init = tf.initialize_all_variables()
    sess.run(init)
    print('\nLearning...')
    for _ in range(10):
        sess.run([updateModel], feed_dict={inputs:np.arange(10)+np.random.random((1,10)), ground_truth:np.arange(4).reshape(1, 4)})
    saver.save(sess,'./tensorflowModel.ckpt')

new_graph = tf.Graph()
with tf.Session(graph=new_graph) as sess:
    inputs, out = make_network(model_phase='test')
    saver = tf.train.import_meta_graph('./tensorflowModel.ckpt.meta')
    saver.restore(sess, tf.train.latest_checkpoint('./'))

    # evaluation
    print('\nEvaluation...')
    for _ in range(10):
        _ = sess.run(out, feed_dict={inputs:np.arange(10).reshape(1,10)})

Answer 1

I don't know why creating an unused Saver makes the problem go away, but the code betrays a misunderstanding.

When you are restoring, you are creating the model graph twice. First, you call make_network() which creates the computation graph and variables. You then also call import_meta_graph which also creates a graph and variables. You should create a saver with simple saver = tf.train.Saver() instead of saver = tf.train.import_meta_graph('./tensorflowModel.ckpt.meta')