关于Theano扫描的一些问题

Question

I am a little confused about theano scan mechanism, here is a simple code snippet to calculate A^k

import numpy
import theano
import theano.tensor as T 

def recurrence(pre_output, a):
    print("test")
    return pre_output*a

x = T.ivector('x')

o, updates = theano.scan(
    fn = recurrence,
    outputs_info = T.ones_like(x),
    non_sequences = x,
    n_steps = 5
)

fun = theano.function([x], o)
print(fun([1,2,3]))

I set print("test") in scan function, since n_steps is 5, the recurrence function should be called 5 times, My first thought was that "test" string should be printed 5 times.

But, as shown in the following output, "test" string just output 1 time.

test
[[  1   2   3]
 [  1   4   9]
 [  1   8  27]
 [  1  16  81]
 [  1  32 243]]

So, a little confused, since recurrence function is called multiple (n_steps) times, why "test" string only output one time?

Any help would be much appreciated.

Thanks

Answer 1

I won't make it too deep.

The mechanism of theano is that it builds a graph for calculation procedure, and does the math according to the graph in a sophisticated way we don't have to care. That's why theano can calculate gradient, because it models the calculation procedure before.

Here comes to the point, in theano.scan(fn=xxx,), by giving scan your fn, you tells theano how to build the graph instead of what to do in this loop .

Theano will building the graph according to your fn but it will calculate it in it's own way instead of using your code.

So here is the conclusion: your code is only used once when scan trying to build the graph, then it's discarded and that explains why your "test" only appear ones.

I hope this helps.

And here is a demo I created to explain, if you like you can try to read it. I come from China and hope my bad English does not make you uneasy.

https://gist.github.com/NickQianFeng/9b91f2ecaa4f7e5ddb89d1b50cac1576