如何防止 TensorFlow eval 梯度

Question

Suppose that I have simple TensorFlow model for MNIST data like this

import tensorflow as tf
from tensorflow.examples.tutorials.mnist import input_data
mnist = input_data.read_data_sets("MNIST_data/", one_hot=True)

x = tf.placeholder(tf.float32, [None, 784]) 
y_ = tf.placeholder(tf.float32, [None, 10])
W = tf.Variable(tf.zeros([784, 10]))
b = tf.Variable(tf.zeros([10]))

y = tf.nn.softmax(tf.matmul(x, W) + b)

cross_entropy = tf.reduce_mean(-tf.reduce_sum(y_ * tf.log(y), reduction_indices=[1]))
train_step = tf.train.GradientDescentOptimizer(0.5).minimize(cross_entropy)
init = tf.initialize_all_variables()
sess = tf.Session()
sess.run(init)

for i in range(1000):
    batch_xs, batch_ys = mnist.train.next_batch(100)
    sess.run(train_step, feed_dict={x: batch_xs, y_: batch_ys})

When training is done, I want to transform test data into output

y_transformed = sess.run(y, feed_dict={x: mnist.test.images})

But, as I understood, doing this cause TensorFlow to calculate gradients for W and b. And this is a lot of overhead in more complex cases. So, how can I avoid this gradient calculations?

Answer 1

Tensor.eval is just a shorthand for Session.run for single tensor, which will not improve performance here.

In the document of Session.run , it says:

This method runs one "step" of TensorFlow computation, by running the necessary graph fragment to execute every Operation and evaluate every Tensor in fetches, substituting the values in feed_dict for the corresponding input values.

And the gradients of variables y is obviously not a necessary fragment. Thus I don't think it will be run unless you do something like train it with optimizers.

Please correct me if I am wrong.