theano定义了重复调用另一个函数的函数？

Question

My training function:

def fit(self, X, y):
    batch_size = 20

    index = T.lscalar()  # index to a [mini]batch
    updates = {}

    return theano.function(
        inputs=[index], outputs=self.cost, updates=updates,
        givens={
            self.sym_X: X[index * batch_size:(index + 1) * batch_size],
            self.sym_y: y[index * batch_size:(index + 1) * batch_size]})

Then from elsewhere:

fn = obj.fit(X, y)
for i in range(10):
    fn(i)

So what I'd like this to look like is

fn = obj.fit(X, y)
fn()

I'm really not even sure how to start on this, as theano is still pretty mind-bending for me. I was able to get this far but loops are seriously challenging.

I have the vague notion that if I can turn the theano.function into a theano.scan, and then put an outer theano.function around it - that might work. However, theano.scan is still magical to me (despite my best efforts).

How can I make it so that the looping over minibatches is incorporated into a single function call?

Update:

I thought I had it! I got this:

def fit(self, X, y):
    batch_size = 20
    n_batches = 5

    index = theano.shared(0)

    ## index to a [mini]batch
    updates = {
        index: index + batch_size
    }

    return theano.function(
        inputs=[], outputs=[self.cost] * n_batches, updates=updates,
        givens={
            index: 0,
            self.sym_X: X[index * batch_size:(index + 1) * batch_size],
            self.sym_y: y[index * batch_size:(index + 1) * batch_size]})

But unfortunately it seems like since I use index to calculate the batches in the givens, I can't also update on it:

Traceback (most recent call last):
  File "skdeeplearn/classifiers/test/test_classifiers.py", line 79, in test_logistic_sgd
    fn = clf.fit(self.shared_X, self.shared_y)
  File "skdeeplearn/classifiers/logistic_sgd.py", line 139, in fit
    self.sym_y: y[index * batch_size:(index + 1) * batch_size]})
  File "/Users/aelaguiz/workspace/pyvotune/venv/lib/python2.7/site-    packages/theano/compile/function.py", line 206, in function
    profile=profile)
  File "/Users/aelaguiz/workspace/pyvotune/venv/lib/python2.7/site-packages/theano/compile/pfunc.py", line 461, in pfunc
    no_default_updates=no_default_updates)
  File "/Users/aelaguiz/workspace/pyvotune/venv/lib/python2.7/site-packages/theano/compile/pfunc.py", line 162, in rebuild_collect_shared
    "to be replaced by %s." % (v_orig, v_repl))
AssertionError: When using 'givens' or 'replace' with several (old_v, new_v) replacement pairs, you can not have a new_v variable depend on an old_v one. For instance, givens = {a:b, b:(a+1)} is not allowed. Here, the old_v <TensorType(int64, scalar)> is used to compute other new_v's, but it is scheduled to be replaced by <TensorType(int64, scalar)>.

---

Update 2:

def fit(self, X, y):
    batch_size = 20
    n_batches = 5

    index = theano.shared(0)

    ## index to a [mini]batch
    updates = {
        index: index + batch_size
    }

    return theano.function(
        inputs=[], outputs=[self.cost] * n_batches, updates=updates,
        givens={
            self.sym_X: X[index * batch_size:(index + 1) * batch_size],
            self.sym_y: y[index * batch_size:(index + 1) * batch_size]})

This actually runs, but it's output is weird:

[array(0.6931471824645996, dtype=float32), array(0.6931471824645996, dtype=float32), array(0.6931471824645996, dtype=float32), array(0.6931471824645996, dtype=float32), array(0.6931471824645996, dtype=float32)]

Everytime I run it I get the same output, even though X & y are initialized to random values each run.

Answer 1

Everybody I know do the loop over minibatch in python. This can be done with scan, but all your tries here did not used scan. So it is normal that they didn't worked. You need to call the scan function somewhere to use it (or its higher level interface like map). In fact, in your case, I think you can use theano.scan(fn, theano.tensor.arange(N)) .

I can't answer all your questions in this post as the snippet of code is incomplete, but here is some information:

return theano.function(
    inputs=[], outputs=[self.cost] * n_batches,

Here: [self.cost] * n_batches is pure python code. This create a list of n_batches element, where each elements are self.cos t. So if n_batches is 3, you will have outputs=[self.cost, self.cost, self.cost] . That is why you got the same value outputed multiple time.

I can't tell you why you always add the same answer as I need information that wasn't provided.