Is everything a Tensor in TensorFlow, including operations?

Question

I've been reading the documentation on core graph structures and it seems there is a disagreement with what TensorFlow actually does and the docs (unless I have a misunderstanding, which I assume I do).

The documentation says there are Operation objects and Tensor objects . It gives examples of such and thus I tried creating some and asking python what types they are. First lets do a constant:

c = tf.constant(1.0)

print c #Tensor("Const_1:0", shape=(), dtype=float32)
print type(c) #<class 'tensorflow.python.framework.ops.Tensor'>

it says its a Tensor. Great! Makes sense and it even gives me information about its contents.

I did the same experiment with what I expected to be an operation:

W = tf.Variable(tf.truncated_normal([784, 10], mean=0.0, stddev=0.1))
b = tf.Variable(tf.constant(0.1, shape=[10]))
Wx = tf.matmul(x, W)
print Wx #Tensor("MatMul:0", shape=(?, 10), dtype=float32)
print type(Wx) #<class 'tensorflow.python.framework.ops.Tensor'>

However, as you can see, Tensor flow said that both Wx and c are the same type. Does this mean that there are no operation objects or am I doing something wrong?

Answer 1

tf.Varible is an tensor, and then you assign a value to it, that is an operation, an assign-operation. Or you use tf.mul() , that also an operation

Answer 2

There are operations. You can get a list of all operations in the graph by graph.get_operations() (where you could get graph via tf.get_default_graph() or sess.graph or whatever is appropriate in your situation).

It's just that, in Python, things like tf.mul return the tensor that the multiplication operation produces (everything else would be annoying since it's tensors that you use as input in further operation).

Answer 3

I am not an expert, but maybe this will clear things up a bit.

x = tf.constant(1, shape=[10, 10])
y = tf.constant(1, shape=[10, 10])
z = tf.matmul(x, y, name='operation')
# print(z)
# tf.Tensor 'operation:0' shape=(10, 10) dtype=int32
# z is a placeholder for the result of multiplication of x and y
# but it has an operation attached to it
# print(z.op)
# tensorflow.python.framework.ops.Operation at 0x10bfe40f0
# and so do x and y
# print(x.op)
# 
ses = tf.InteractiveSession()
# now that we are in a session, we have an operation graph
ses.graph.get_operation_by_name('operation')
# tensorflow.python.framework.ops.Operation at 0x10bfe40f0
# same operation that we saw attached to z
ses.graph.get_operations()
# shows a list of three operations, just as expected
# this way you can define the graph first and then run all the operations in a session

Answer 4

I'm not very familiar with TensorFlow, but it seems to be a basic concept that Wx is a symbolic handle for the output of tf.matmul(x, W) . So you actually created an Operation, but accessing Wx will give you a representation of the result (even though it's not calculated until you run a session).

Take a look at the TensorFlow FAQ for a more detailed explanation.

Answer 5

Think about this outside the context of tensorflow, and just in base python. Let's say you do this:

def f(x):
    return(x+1)

x = 0

print(type(x))
print(type(f(x)))

You get int in both cases, right? But what if you do

type(f)

In this case, you get a function . Same with tensorflow: the type of the result of an operation is a new tensor, but the type of the operation itself is not a tensor.