如何在tensorflow中打印局部张量？

Question

I want to print a tensor in my program to see its internal values once it gets evaluated. The problem, however, is that the tensor being declared inside a function. To understand my problem better, here is some example code to better explain what it is I want to do:

a = tf.Variable([[2,3,4], [5,6,7]])
b = tf.Variable([[1,2,2], [3,3,3]])

def divide(a,b):
    with tf.variable_scope('tfdiv', reuse=True):
        c = tf.divide(a,b, name='c')
    # Cannot print(c) here, as this will only yield tf info on c
    return c

d = divide(a,b)

with tf.Session() as sess:
    sess.run(tf.global_variables_initializer())
    sess.run(d)
    sess.run(tf.get_variable('tfdiv/c:0').eval(session=sess))

Previously, I have been able to do a print(c.eval(session=sess)), but as c is a local variable inside a function now, that does not work. As can be seen in the code above, I have tried to use tensorflow's variable scope in order to access the variable and then evaluate it. Unfortunately, this results in the error message:

ValueError: Shape of a new variable (tfdiv/c:0) must be fully defined, but 
instead was <unknown>.

I tried to use the reuse=True flag, but I still get the same error. Any thoughts on how I can solve this problem? Best would be if there is a print(c) equivalent that can be put into the divide function, as written in the code above.

Answer 1

This will achieve what you want to do:

with tf.Session() as sess:
    sess.run(tf.global_variables_initializer())
    print(sess.run(d))

Alternatively, you could replace the last line with:

print(sess.run(tf.get_default_graph().get_tensor_by_name('tfdiv/c:0')))

Answer 2

It is important to understand the difference between Python side code and TensorFlow side code. In python, you only setup the graph: d = divide(a, b) creates something like:

You set up a node (square) that will divide the data in the nodes a and b . It doesn't divide them right away! Note that in black you have the python variable names, and in gray you have the TensorFlow node names ¹ . a and b also have some default names, if you didn't specify them. The gray "c" you specified with name='c' . And local variable c and global variable d (Python) both refer to that same operation (node).

This is why if you say print(d) merely prints the info about that node. Once you setup the graph, doing sess.run(d) runs all the nodes required by the node in d on TensorFlow side . Then it retrieves the result and makes it available on python side as a numpy array.

You can use tf.Print(input, data) to print tensors on TF side . Note this is an operation (a node in the graph) that does nothing to the input tensor, it merely passes it through, while also printing everything in data .

In your case, you can use Print on tensorflow side it like this:

def divide(a,b):
    with tf.variable_scope('tfdiv', reuse=True):
        c = tf.divide(a,b, name='c')
        cp = tf.Print(c, [c], message='Value of c: ', name='P')

    return cp

This effectively adds another node in the graph (named P on TF side):

Now the value of operation c will be printed every time it will be evaluated. Note it will also be printed every time one of its dependencies will be evaluated, for example if you later do e = d + 1 , when you evaluate e , it needs d , which refers to the printing node (returned from the function divide ).

Finally, note that if you do this in a Jupyter notebook, the print will appear in the terminal of the notebook server. The details of this are not important for now :).

¹ the :0 is added by default so that you can retrieve any tensor with by using name_of_op:0 . Distinction between name of operation ( tfdiv/c ) and name of tensor( tfdiv/c:0 ).