I am a theano newbie.
Can someone please explain the following code?
from theano.tensor.shared_randomstreams import RandomStreams
from theano import function
srng = RandomStreams(seed=234)
rv_u = srng.uniform((2,2))
rv_n = srng.normal((2,2))
f = function([], rv_u)
g = function([], rv_n, no_default_updates=True) #Not updating rv_n.rng
nearly_zeros = function([], rv_u + rv_u - 2 * rv_u)
state_after_v0 = rv_u.rng.get_value().get_state()
nearly_zeros() # this affects rv_u's generator
v1 = f()
rng = rv_u.rng.get_value(borrow=True)
rng.set_state(state_after_v0)
rv_u.rng.set_value(rng, borrow=True)
v2 = f() # v2 != v1
v3 = f() # v3 == v1
Q1. How does nearly_zeros() affect rv_u's generator?
Q2. Why ?
v2 != v1
v3 == v1
Q1
Looks like only 1 "value" (ie a 2x2 matrix) is being generated by rv_u's generator. You can see this if you use theano.printing.debugprint
to print the graph of the function. For reference this is what I got:
>>> theano.printing.debugprint(nearly_zeros)
Elemwise{Composite{((i0 + i0) - (i1 * i0))}}[(0, 0)] [@A] '' 1
|RandomFunction{uniform}.1 [@B] '' 0
| |<RandomStateType> [@C]
| |TensorConstant{(2,) of 2} [@D]
| |TensorConstant{0.0} [@E]
| |TensorConstant{1.0} [@F]
|TensorConstant{(1, 1) of 2.0} [@G]
RandomFunction{uniform}.0 [@B] '' 0
This means the two functions (nearly_zero and f) both grab only 1 value from rv_u, hence why
v3 == v1
Q2
Theano is largely a symbolic math package. You define relationships between symbolic variables and Theano figures out how to evaluate those relationships.
So you could consider rv_u to be a variable which represents a single 2x2 matrix, rather than a rng which generates a new 2x2 matrix each time it's "called." Given that view, Theano would only need to call the underlying rng once to get the value for the variable rv_u.
When nearly_zeros()
is called, it uses the current rng state of the random stream to return a new rv_u
and increments the state just as it does for the function f
. The easiest way to see what is happening to rv_u
is to add it as an output to the nearly_zeros()
function. Although rv_u
is present three times, the value of rv_u
is sampled from the random stream only once, hence nearly_zero
will indeed be nearly zero (up to floating-point quantization error.)
nearly_zeros = function([], [rv_u, rv_u + rv_u - 2 * rv_u])
f() # The return value will be different from the rv_u in the list above
If you do not intend for it to increment, you can specify the no_default_updates=True
as was done for g
.
nearly_zeros = function([], [rv_u, rv_u + rv_u - 2 * rv_u],
no_default_updates=True)
f() # The return value will be equal to the rv_u in the list above
The "why" here was a little ambiguous, so I present two possibilities with the answers.
nearly_zeros()
impact rv_u
in this way?
rv_u
is a uniform srng object which uses a shared variable that is incremented each time it is requested, unless specifically told otherwise (by means of the no_defalut_updates
parameter.) This is true whether f
or nearly_zero
happens to be the function that requests the value of rv_u
.
from theano.tensor.shared_randomstreams import RandomStreams
from theano import function
srng = RandomStreams(seed=234)
rv_u = srng.uniform((2,2))
rv_n = srng.normal((2,2))
f = function([], rv_u)
g = function([], rv_n, no_default_updates=True) #Not updating rv_n.rng
nearly_zeros = function([], [rv_u, rv_u + rv_u - 2 * rv_u]) # Give rv_u, too
state_after_v0 = rv_u.rng.get_value().get_state()
nearly_zeros() # this affects rv_u's generator
v1 = f()
rng = rv_u.rng.get_value(borrow=True)
rng.set_state(state_after_v0)
rv_u.rng.set_value(rng, borrow=True)
v2 = f() # v2 != v1, but equal to rv_u used by nearly_zeros
v3 = f() # v3 == v1
This is because your code saves the state after defining nearly_zeros
but before it is called. When the state is set, the first value that is returned is the value used by nearly_zeros
(which is equal to v2
.) The next value requested will be equal to the value of v3
. The following is a nearly identical copy of your code, and it should prove illustrative:
from theano.tensor.shared_randomstreams import RandomStreams from theano import function srng = RandomStreams(seed=234) rv_u = srng.uniform((2,2)) rv_n = srng.normal((2,2)) f = function([], rv_u) g = function([], rv_n, no_default_updates=True) #Not updating rv_n.rng nearly_zeros = function([], [rv_u, rv_u + rv_u - 2 * rv_u]) # Give rv_u, too state_after_v0 = rv_u.rng.get_value().get_state() nearly_zeros() # this affects rv_u's generator v1 = f() rng = rv_u.rng.get_value(borrow=True) rng.set_state(state_after_v0) rv_u.rng.set_value(rng, borrow=True) v2 = f() # v2 != v1, but equal to rv_u used by nearly_zeros v3 = f() # v3 == v1
Aside: Although this thread is a bit old, hopefully someone will find these answers helpful.
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.