在theano函数的给定参数中切片矩阵

Question

I have the following piece of code, in which I attempt to apply PCA to the MNIST dataset.

X_train, y_train = mnist.data[:60000] / 255., mnist.target[:60000]

X_train, y_train = shuffle(X_train, y_train)
X_train, y_train = X_train[:5000], y_train[:5000]

pca = PCA(M=2)

X = T.matrix('X', dtype='float64')
i = T.scalar()
j = T.scalar()

# Theano function which fits the model to the
# data i.e. applies dimensionality reduction
transform = theano.function(
   inputs=[i, j],
   outputs=pca.transform(X),
   givens={
        X: X_train[(y_train == i) + (y_train == j)]
   }
)

X_transformed = transform(i, j)
y_ = y_train[(y_train == i) + (y_train == j)]

As can be seen in the code above, I try to substitute X with a slice of the training dataset in the givens params. The above code, however, results in the following error:

TypeError: Cannot convert Type TensorType(float64, vector) (of Variable <TensorType(float64, vector)>) into Type TensorType(float64, matrix). You can try to manually convert <TensorType(float64, vector)> into a TensorType(float64, matrix).

Which implies that I am trying to assign a vector to a matrix, which isn't the behaviour I would expect (I doubled checked using numpy). I also tried a different approach, whereby I index the training dataset X_train with the array of booleans directly instead of using indices are performing the slices my self, but that also didn't work.

X_train, y_train = mnist.data[:60000] / 255., mnist.target[:60000]

X_train, y_train = shuffle(X_train, y_train)
X_train, y_train = X_train[:5000], y_train[:5000]

pca = PCA(M=2)

X = T.matrix('X', dtype='float64')
bool_arr = T.vector()

# Theano function which fits the model to the
# data i.e. applies dimensionality reduction
transform = theano.function(
   inputs=[i, j],
   outputs=pca.transform(X),
   givens={
        X: X_train[bool_arr]
   }
)

X_transformed = transform((y_train == i) + (y_train == j))
y_ = y_train[(y_train == i) + (y_train == j)]

Which gives the following error:

IndexError: only integers, slices (`:`), ellipsis (`...`), numpy.newaxis (`None`) and integer or boolean arrays are valid indices

The only approach that did work is using disregarding the givens param, and using only the inputs and outputs, as such:

X = T.matrix('X', dtype='float64')

# Theano function which fits the model to the
# data i.e. applies dimensionality reduction
transform = theano.function(
    inputs=[X],
    outputs=pca.transform(X),
)

X_ = X_train[(y_train == i) + (y_train == j)]
y_ = y_train[(y_train == i) + (y_train == j)]

X_transformed = transform(X_)

None the less, I feel curious as to why my first two approaches do not work, and would appreciate if any one could point me to where I might be going wrong, since I am just starting with Theano.

Answer 1

The theano documentation states:

givens (iterable over pairs (Var1, Var2) of Variables. List, tuple or dict. The Var1 and Var2 in each pair must have the same Type.) – specific substitutions to make in the computation graph ( Var2 replaces Var1 ).

And in the tutorial examples , there is the statement (emphasis mine)

In practice, a good way of thinking about the givens is as a mechanism that allows you to replace any part of your formula with a different expression that evaluates to a tensor of same shape and dtype .

So, you cannot replace a matrix by a vector by means of the givens parameter since they don't have the same shape.