[英]Tensorflow: Why must `saver = tf.train.Saver()` be declared after variables are declared?
Important clarification: I was only running this section, the graph definition, in a notebook enviroment. 重要说明:我只在笔记本环境中运行此部分,图形定义。 I had not run an actual session yet.
我还没有参加过实际的会议。
When running this code: 运行此代码时:
with graph.as_default(): #took out " , tf.device('/cpu:0')"
saver = tf.train.Saver()
valid_examples = np.array(random.sample(range(1, valid_window), valid_size)) #put inside graph to get new words each time
train_dataset = tf.placeholder(tf.int32, shape=[batch_size, cbow_window*2 ])
train_labels = tf.placeholder(tf.int32, shape=[batch_size, 1])
valid_dataset = tf.constant(valid_examples, dtype=tf.int32)
valid_datasetSM = tf.constant(valid_examples, dtype=tf.int32)
embeddings = tf.get_variable( 'embeddings',
initializer= tf.random_uniform([vocabulary_size, embedding_size], -1.0, 1.0))
softmax_weights = tf.get_variable( 'softmax_weights',
initializer= tf.truncated_normal([vocabulary_size, embedding_size],
stddev=1.0 / math.sqrt(embedding_size)))
softmax_biases = tf.get_variable('softmax_biases',
initializer= tf.zeros([vocabulary_size]), trainable=False )
embed = tf.nn.embedding_lookup(embeddings, train_dataset) #train data set is
embed_reshaped = tf.reshape( embed, [batch_size*cbow_window*2, embedding_size] )
segments= np.arange(batch_size).repeat(cbow_window*2)
averaged_embeds = tf.segment_mean(embed_reshaped, segments, name=None)
#return tf.reduce_mean( tf.nn.sampled_softmax_loss(weights=softmax_weights, biases=softmax_biases, inputs=averaged_embeds,
#labels=train_labels, num_sampled=num_sampled, num_classes=vocabulary_size))
loss = tf.reduce_mean(
tf.nn.sampled_softmax_loss(weights=softmax_weights, biases=softmax_biases, inputs=averaged_embeds,
labels=train_labels, num_sampled=num_sampled, num_classes=vocabulary_size))
norm = tf.sqrt(tf.reduce_sum(tf.square(embeddings), 1, keepdims=True))
normSM = tf.sqrt(tf.reduce_sum(tf.square(softmax_weights), 1, keepdims=True))
normalized_embeddings = embeddings / norm
normalized_embeddingsSM = softmax_weights / normSM
valid_embeddings = tf.nn.embedding_lookup(
normalized_embeddings, valid_dataset)
valid_embeddingsSM = tf.nn.embedding_lookup(
normalized_embeddingsSM, valid_datasetSM)
similarity = tf.matmul(valid_embeddings, tf.transpose(normalized_embeddings))
similaritySM = tf.matmul(valid_embeddingsSM, tf.transpose(normalized_embeddingsSM))
I got this error 我收到了这个错误
ValueError: No variables to save ValueError:没有要保存的变量
while pointing to this line 同时指着这条线
saver = tf.train.Saver()
I searched stack overflow and found this answer 我搜索了堆栈溢出并找到了这个答案
Tensorflow ValueError: No variables to save from Tensorflow ValueError:无需保存的变量
So I simply put that line at the bottom of the graph definition like so 所以我只是简单地将该行放在图形定义的底部
with graph.as_default(): #took out " , tf.device('/cpu:0')"
valid_examples = np.array(random.sample(range(1, valid_window), valid_size)) #put inside graph to get new words each time
train_dataset = tf.placeholder(tf.int32, shape=[batch_size, cbow_window*2 ])
train_labels = tf.placeholder(tf.int32, shape=[batch_size, 1])
valid_dataset = tf.constant(valid_examples, dtype=tf.int32)
valid_datasetSM = tf.constant(valid_examples, dtype=tf.int32)
embeddings = tf.get_variable( 'embeddings',
initializer= tf.random_uniform([vocabulary_size, embedding_size], -1.0, 1.0))
softmax_weights = tf.get_variable( 'softmax_weights',
initializer= tf.truncated_normal([vocabulary_size, embedding_size],
stddev=1.0 / math.sqrt(embedding_size)))
softmax_biases = tf.get_variable('softmax_biases',
initializer= tf.zeros([vocabulary_size]), trainable=False )
embed = tf.nn.embedding_lookup(embeddings, train_dataset) #train data set is
embed_reshaped = tf.reshape( embed, [batch_size*cbow_window*2, embedding_size] )
segments= np.arange(batch_size).repeat(cbow_window*2)
averaged_embeds = tf.segment_mean(embed_reshaped, segments, name=None)
loss = tf.reduce_mean(
tf.nn.sampled_softmax_loss(weights=softmax_weights, biases=softmax_biases, inputs=averaged_embeds,
labels=train_labels, num_sampled=num_sampled, num_classes=vocabulary_size))
norm = tf.sqrt(tf.reduce_sum(tf.square(embeddings), 1, keepdims=True))
normSM = tf.sqrt(tf.reduce_sum(tf.square(softmax_weights), 1, keepdims=True))
normalized_embeddings = embeddings / norm
normalized_embeddingsSM = softmax_weights / normSM
valid_embeddings = tf.nn.embedding_lookup(
normalized_embeddings, valid_dataset)
valid_embeddingsSM = tf.nn.embedding_lookup(
normalized_embeddingsSM, valid_datasetSM)
similarity = tf.matmul(valid_embeddings, tf.transpose(normalized_embeddings))
similaritySM = tf.matmul(valid_embeddingsSM, tf.transpose(normalized_embeddingsSM))
saver = tf.train.Saver()
And then there were no errors! 然后没有错误!
Why is this so? 为什么会这样? The graph definition is only defining the graph, not running anything.
图形定义仅定义图形,而不是运行任何图形。 Perhaps it's an bug prevention measure?
也许这是一个防止错误的措施?
It does not have to. 它不必。
tf.train.Saver
has a defer_build
argument that, if set to True
, allows you to define variables after it has been constructed. tf.train.Saver
有一个defer_build
参数,如果设置为True
,则允许您在构造变量后定义它们。 You then need to call build
explicitly though. 然后,您需要显式调用
build
。
saver = tf.train.Saver(defer_build=True)
# construct your graph, create variables...
...
saver.build()
graph.finalize()
# go on with training
From the documentation on tf.train.Saver the __init__
method has a parameter var_list
with the description: 从tf.train.Saver上的文档中,
__init__
方法有一个参数var_list
其中包含以下描述:
var_list: A list of Variable/SaveableObject, or a dictionary mapping names
to SaveableObjects. If None, defaults to the list of all saveable objects.
This suggests that the saver makes a list of variables to save when it's first created, which by default contains all of the variables it can find. 这表明保护程序会在首次创建时创建要保存的变量列表,默认情况下包含它可以找到的所有变量。 If no variables have been made, an error makes sense since there are no variables to save.
如果没有变量,则错误是有意义的,因为没有要保存的变量。
Random examples: 随机例子:
import tensorflow as tf
saver = tf.train.Saver()
The above throws an error, and so does below 以上引发了错误,下面也是如此
import tensorflow as tf
x = tf.placeholder(dtype=tf.float32)
saver = tf.train.Saver()
But this last example runs, 但最后一个例子运行,
import tensorflow as tf
x = tf.Variable(0.0)
saver = tf.train.Saver()
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.