[英]ValueError: Tensor(“BN_1/moments/Squeeze:0”, shape=(32, 256, 32), dtype=float32) must be from the same graph as Tensor
I'm trying to get started with TensorFlow in python, building a simple CNN with batch normalization. 我正在尝试使用python中的TensorFlow入门,使用批处理规范化构建简单的CNN。 But when i create a new graph to run, exception happens to BN.
但是当我创建一个新的图形来运行时,BN会发生异常。
My key codes is as follows 我的关键代码如下
**# exception here**
def batch_norm(x, beta, gamma, phase_train, scope='bn', decay=0.9, eps=1e-5):
with tf.variable_scope(scope):
batch_mean, batch_var = tf.nn.moments(x, [0], name='moments')
ema = tf.train.ExponentialMovingAverage(decay=decay)
def mean_var_with_update():
ema_apply_op = ema.apply([batch_mean, batch_var])
with tf.control_dependencies([ema_apply_op]):
return tf.identity(batch_mean), tf.identity(batch_var)
mean, var = tf.cond(phase_train, mean_var_with_update, lambda: (ema.average(batch_mean), ema.average(batch_var)))
normed = tf.nn.batch_normalization(x, mean, var, beta, gamma, eps)
return normed
training code: 训练代码:
# start training
output = conv2d_net()
loss = tf.reduce_mean(tf.nn.sigmoid_cross_entropy_with_logits(logits=output, labels=Y))
optimizer = tf.train.AdamOptimizer(learning_rate=0.002).minimize(loss)
predict = tf.reshape(output, [-1, MAX_CAPTCHA, CHAR_SET_LEN])
max_idx_p = tf.argmax(predict, 2)
max_idx_l = tf.argmax(tf.reshape(Y, [-1, MAX_CAPTCHA, CHAR_SET_LEN]), 2)
correct_pred = tf.equal(max_idx_p, max_idx_l)
accuracy = tf.reduce_mean(tf.cast(correct_pred, tf.float32))
with tf.Session() as sess:
sess.run(tf.global_variables_initializer())
step = 0
while True:
batch_x, batch_y = get_next_batch(64)
_, loss_ = sess.run([optimizer, loss],
feed_dict={X: batch_x, Y: batch_y, keep_prob: 0.75, train_phase: True})
print(step, loss_)
if step % 10 == 0 and step != 0:
batch_x_test, batch_y_test = get_next_batch(100)
acc = sess.run(accuracy,
feed_dict={X: batch_x_test, Y: batch_y_test, keep_prob: 1., train_phase: False})
print("step %s,accuracy:%s" % (step, acc))
if acc > 0.05:
# stop training and save parameters in layer
result_weights['wc1'] = weights['wc1'].eval(sess)
...
break
step += 1
Create new graph for exporting: 创建要导出的新图形:
EXPORT_DIR = './model'
if os.path.exists(EXPORT_DIR):
shutil.rmtree(EXPORT_DIR)
g = tf.Graph()
with g.as_default():
x_2 = tf.placeholder(tf.float32, shape=[None, IMAGE_HEIGHT * IMAGE_WIDTH], name="input")
x_image = tf.reshape(x_2, shape=[-1, IMAGE_HEIGHT, IMAGE_WIDTH, 1])
# fill trained parameters and create new cnn layers
WC1 = tf.constant(result_weights['wc1'], name="WC1")
...
**# crash here!!!**
CONV1 = conv2d(WC1, BC1, x_image, tf.constant(0.0, shape=[32]),
tf.random_normal(shape=[32], mean=1.0, stddev=0.02), scope='BN_1')
OUTPUT = tf.add(tf.matmul(FULL1, W_OUT), B_OUT)
OUTPUT = tf.nn.sigmoid(OUTPUT, name="output")
sess = tf.Session()
sess.run(tf.global_variables_initializer())
graph_def = g.as_graph_def()
tf.train.write_graph(graph_def, EXPORT_DIR, 'phone_model_graph.pb', as_text=True)
I create a new graph at last. 我终于创建了一个新图。 The exception means it uses incorrect parameter in old training graph.
异常表示它在旧的训练图中使用了错误的参数。 How to explain it?
怎么解释呢?
Thank you very much! 非常感谢你!
I call batch_norm in fuction conv2d. 我在函数conv2d中调用batch_norm。 It seems no tensor passed to the new graph.
似乎没有张量传递给新图。
def conv2d(w, b, x, tf_constant, tf_random_normal, scope, keep_p=1., phase=tf.constant(False)):
out = tf.nn.bias_add(tf.nn.conv2d(x, w, strides=[1, 1, 1, 1], padding='SAME'), b)
out = batch_norm(out, tf_constant, tf_random_normal, phase, scope=scope)
out = tf.nn.relu(out)
out = tf.nn.max_pool(out, ksize=[1, 2, 2, 1], strides=[1, 2, 2, 1], padding='SAME')
out = tf.nn.dropout(out, keep_p)
return out
I create a new graph at last.
我终于创建了一个新图。
That's the key statement here: upon creation of a new graph one can't use any tensor from the old graph. 这就是这里的关键声明:创建新图后,您将无法使用旧图的任何张量。 See a detailed explanation in this question .
请参阅此问题的详细说明。 According to the stacktrace, at least one of the tensors that is passed to the
batch_norm
is defined before g.as_default()
, that's why tensorflow crashes. 根据stacktrace,在
g.as_default()
之前定义了至少一个传递给batch_norm
的张量,这就是tensorflow崩溃的原因。 From your code snippets it's unclear how exactly the batch_norm
is called, so I can't say which one. 从您的代码片段中,还不清楚如何精确地调用
batch_norm
,因此我无法说出哪个。
You can check this hypothesis by printing x.graph
and g
and checking if these values are different. 您可以通过打印
x.graph
和g
并检查这些值是否不同来检查该假设。 In order to avoid this problem you can either do all the work inside one graph (which is a recommended way) or define both graphs in different python scopes thus making impossible to accidentally reuse the same python variable in two graphs. 为了避免此问题,您可以在一个图内完成所有工作(推荐的方法),也可以在不同的python范围内定义两个图,从而避免意外地在两个图中重用同一python变量。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.