当我向theano函数中的给定参数发送numpy数组时，为什么会出现此Theano TypeError

Question

I am new to theano, and am still struggling with the"pseudo code" style of theano on the one hand and strict type checking on the other. I am more of a C programmer and a python programmer. Can someone please point out where I am going wrong in this example code which uses mean square error between predicted y points and training y points for x values, to get the optimal slope and intercept of a linear fit?

The code is below:

import numpy as np
import theano
import theano.tensor as T
from collections import OrderedDict

class LinearModel:
    def __init__(self,num_points):
        self.m = theano.shared(value=0.1,name='m')
        self.b = theano.shared(value=1, name='b')
        self.params = [self.m, self.b]

        def step(x_t):
            y_t = self.m * x_t + self.b
            return y_t

        self.x = T.matrix('x',dtype=theano.config.floatX)
        self.y, _ = theano.scan(
                        fn=step,
                        sequences=self.x,
                    ) 

        self.loss = lambda y_train: self.mse(y_train)

    def mse(self, y_train):
        return T.mean((self.y - y_train) ** 2)

    def fit(self,x, y, learning_rate=0.01, num_iter=100):
        trainset_x = theano.tensor._shared(x.astype(np.dtype(np.float32)),borrow=True)
        trainset_y = theano.tensor._shared(y.astype(np.dtype(np.float32)),borrow=True)
        n_train = trainset_x.get_value(borrow=True).shape[0]

        cost = self.loss(trainset_y)
        gparams = T.grad(cost,self.params)

        l_r = T.scalar('l_r', dtype=theano.config.floatX)

        updates = OrderedDict()
        for param,gparam in zip(self.params,gparams):
            updates[param] = param - l_r * gparam

        self.train_model = theano.function(  inputs=[l_r],
                                        outputs=[cost,self.y],
                                        updates=updates,
                                        givens={
                                              self.x: trainset_x,
                                            }
                                        )

        epoch = 0
        while epoch < num_iter:
            cost, _ = self.train_model(learning_rate)
            m = self.m.get_value()
            b = self.b.get_value()
            print "epoch: ",epoch," cost: ",cost," m: ",m," b: ",b


if __name__ == '__main__':
    lin = LinearModel(10)
    x = np.arange(10)
    y = np.random.rand(10)
    lin.fit(x,y,learning_rate=0.01,num_iter=100)

---

The error is:

Traceback (most recent call last): File "~/EclipseWorkspace/MemoryNetworkQA.Theano/linear_regression.py", line 70, in lin.fit(x,y,learning_rate=0.01,num_iter=100) File "~/EclipseWorkspace/MemoryNetworkQA.Theano/linear_regression.py", line 54, in fit self.x: trainset_x, File "/usr/local/lib/python2.7/dist-packages/theano/compile/function.py", line 266, in function profile=profile) File "/usr/local/lib/python2.7/dist-packages/theano/compile/pfunc.py", line 489, in pfunc no_default_updates=no_default_updates) File "/usr/local/lib/python2.7/dist-packages/theano/compile/pfunc.py", line 217, in rebuild_collect_shared raise TypeError(err_msg, err_sug)

TypeError: ('An update must have the same type as the original shared variable (shared_var=b, shared_var.type=TensorType(int64, scalar), update_val=Elemwise{sub,no_inplace}.0, update_val.type=TensorType(float64, scalar)).', 'If the difference is related to the broadcast pattern, you can call the tensor.unbroadcast(var, axis_to_unbroadcast[, ...]) function to remove broadcastable dimensions.')

Answer 1

This code won't execute until the following problems have been addressed.

1. The error reported in the question is due to the type of self.b not matching the type of the update for self.b . self.b has no type specified so one has been inferred. The initial value is a Python integer so the inferred type is an int64 . The update is a floatX because the learning rate is a floatX . You can't update an int64 with a floatX . The solution is to make the initial value a Python float resulting in an inferred floatX type. Change self.b = theano.shared(value=1, name='b') to self.b = theano.shared(value=1., name='b') (note the decimal point after the 1 ).

2. The next problem is that self.x is defined as a matrix but the value passed in the function call in the last line is a vector. A solution is to reshape x into a matrix, eg change x = np.arange(10) to x = np.arange(10).reshape(1,10) .

3. The trainset shared variables have type float32 but this conflicts with other areas of the code that is working with floatX . If your floatX=float32 then there should be no problem but it would be safer to simply use floatX to maintain the same float type throughout. Change trainset_x = theano.tensor._shared(x.astype(np.dtype(np.float32)),borrow=True) to trainset_x = theano.tensor._shared(x.astype(theano.config.floatX),borrow=True) and similarly for trainset_y .

4. The number of epochs is not currently having any effect because epoch is not being incremented. Change while epoch < num_iter: to for epoch in xrange(num_iter): and remove epoch = 0 .

In addition,

• The parameters look like they're not updating but this is a false view. The iterations pass quickly and never stop because of problem 4 above and the learning rate is large enough to make the model converge very quickly. Try changing the learning rate to something much smaller, eg 0.0001, and look at the output for only the first 100 epochs.

• I'd recommend avoiding the use of theano.tensor._shared unless you really do need to force the shared variable to be allocated on the CPU when device=gpu . The preferred method is theano.shared .

• The n_train variable isn't used anywhere.

• You're using givens inconsistently. I'd recommend using it for both x and y , or for neither. Take a look at the logistic regression tutorial for more pointers on this.

• The Theano function is being recompiled on every call to fit but you'd be better off compiling it only once and reusing it on each fit .

• This model can be implemented without using scan . In general, scan is often only needed when the output of a step is a function of the output from an earlier step. scan is also generally much slower than alternatives and should be avoided when possible. You can remove scan by using self.y = self.m * self.x + self.b instead.

• If you do use scan, it's good practice to enable strict mode, via strict=True in the scan call.

• It's good practice to explicitly provide types for all shared variables. You do this for trainset_x and trainset_y but not for self.m and self.b .

Answer 2

Okay, I found that the problem was really in the self.b. After initializing it with an explicit float, the type error goes away.

But the slope and intercepts (self.m and self.b) which are still theano shared variables and are being passed in through updates, are not really getting updated. If anyone can tell me why, it will be a great help. Thanks.

import numpy as np
import theano
import theano.tensor as T
from collections import OrderedDict

class LinearModel:
    def __init__(self,num_points):
        self.m = theano.shared(value=0.1,name='m')
        self.b = theano.shared(value=1.0, name='b')
        self.params = [self.m, self.b]

        def step(x_t):
            y_t = self.m * x_t + self.b
            return y_t

        #self.x = T.matrix('x',dtype=theano.config.floatX)
        #self.x = T.dmatrix('x')
        self.x = T.vector('x',dtype=theano.config.floatX)
        self.y, _ = theano.scan(
                        fn=step,
                        sequences=self.x,
                    ) 

        self.loss = lambda y_train: self.mse(y_train)

    def mse(self, y_train):
        return T.mean((self.y - y_train) ** 2)

    def fit(self,x, y, learning_rate=0.01, num_iter=100):
        trainset_x = theano.tensor._shared(x.astype(np.dtype(np.float32)),borrow=True)
        trainset_y = theano.tensor._shared(y.astype(np.dtype(np.float32)),borrow=True)
        n_train = trainset_x.get_value(borrow=True).shape[0]

        cost = self.loss(trainset_y)
        gparams = T.grad(cost,self.params)

        l_r = T.scalar('l_r', dtype=theano.config.floatX)

        updates = OrderedDict()
        for param,gparam in zip(self.params,gparams):
            updates[param] = param - l_r * gparam

        self.train_model = theano.function(  inputs=[l_r],
                                        outputs=[cost,self.y],
                                        updates=updates,
                                        givens={
                                              self.x: trainset_x,
                                            }
                                        )


        epoch = 0
        while epoch < num_iter:
            cost, _ = self.train_model(learning_rate)
            m = self.m.get_value()
            b = self.b.get_value()
            print "epoch: ",epoch," cost: ",cost," m: ",m," b: ",b
            epoch += 1


if __name__ == '__main__':
    lin = LinearModel(10)
    x = np.array([1.0,2.0,3.0,4.0,5.0,6.0,7.0,8.0,9.0,10.0])
    y = np.random.rand(10)
    lin.fit(x,y,learning_rate=0.01,num_iter=100)