Broadcasting np.dot vs tf.matmul for tensor-matrix multiplication (Shape must be rank 2 but is rank 3 error)

Question

Let's say I have the following tensors:

X = np.zeros((3,201, 340))
Y = np.zeros((340, 28))

Making a dot product of X, Y is successful with numpy, and yields a tensor of shape (3, 201, 28). However with tensorflow I get the following error: Shape must be rank 2 but is rank 3 error ...

minimal code example:

X = np.zeros((3,201, 340))
Y = np.zeros((340, 28))
print(np.dot(X,Y).shape) # successful (3, 201, 28)
tf.matmul(X, Y) # errornous

Any idea how to achieve the same result with tensorflow?

Answer 1

Since, you are working with tensors , it would be better (for performance) to use tensordot there than np.dot . NumPy allows it (numpy.dot) to work on tensors through lowered performance and it seems tensorflow simply doesn't allow it.

So, for NumPy, we would use np.tensordot -

np.tensordot(X, Y, axes=((2,),(0,)))

For tensorflow , it would be with tf.tensordot -

tf.tensordot(X, Y, axes=((2,),(0,)))

Related post to understand tensordot .

Answer 2

Tensorflow doesn't allow for multiplication of matrices with different ranks as numpy does.

To cope with this, you can reshape the matrix. This essentially casts a matrix of, say, rank 3 to one with rank 2 by "stacking the matrices" one on top of the other.

You can use this: tf.reshape(tf.matmul(tf.reshape(Aijk,[i*j,k]),Bkl),[i,j,l])

where i, j and k are the dimensions of matrix one and k and l are the dimensions of matrix 2.

Taken from here .