Multiplying specific dimension matrices

Question

Say I have matrices A and B.

A is a three dimensional array/tensor(?).

[1,2,3,4]
[5,6,7,8]
[1,2,3,4]
[5,6,7,8]

There are say 4 DIFFERENT 2d matrices like the one above across the third dimension.

B is a matrix.

[1,2,3,4]

There are also 4 of of these in B, each one is the SAME.

How would I multiply each vector(?) in B with each 2d matrix in A.

[1,2,3,4]*[1,2,3,4]*[1;2;3;4]
          [5,6,7,8]
          [1,2,3,4]
          [5,6,7,8]

There would be four of the above type multiplications to get four 4x1 vectors. I've tried it with numpy as:

y = numpy.arange(4).reshape(1,4)
z = numpy.arange(64).reshape(4,4,4)
y.dot(z).dot(numpy.transpose(y))
------
Output:
array([[[ 420],
        [ 996],
        [1572],
        [2148]]])

And it works as I want it to. But I have no idea how numpy is broadcasting and I want to know for learning purposes and also other packages for dealing with matrices in different libraries handle broadcasting differently. I've tried to tile B in different ways to achieve the same results, but nothing works. If I'm not explaining anything clearly, let me know.

Also would prefer to get 4x1 rather than 3d return from numpy.

Answer 1

You may want to look into np.einsum . As an example:

>>> mat = np.arange(80).reshape(4, 4, 5)
>>> vec = np.arange(12).reshape(3, 4)
>>> np.einsum('ij,jkl,ik->il', vec, mat, vec)
array([[ 2100,  2136,  2172,  2208,  2244],
       [20900, 21384, 21868, 22352, 22836],
       [58900, 60344, 61788, 63232, 64676]])

If I didn't get the indices wrong, I have 5 matrices of shape 4x4, and 3 vectors of length 4, and am computing the 3x5 quadratic forms of each matrix with each vector.

Answer 2

Note that np.dot behaves differently with different dimension arrays, from here :

For 2-D arrays it is equivalent to matrix multiplication,

for 1-D arrays to inner product of vectors (without complex conjugation).

For N dimensions it is a sum product over the last axis of a and the second-to-last of b

In your example z has 3 dimensions.

Also, there is an important distinction between a numpy array and a numpy matrix :

You are using arrays in your example. Numpy matrices are 2D only and matrix multiplication is achived with the * operator. (With arrays this perfroms element by element multiplication)

Answer 3

To keep the dimensions straight I'm going to use variables. Often in testing I like to use different sizes in each dimension (eg z=...reshape(3,4,5) ), so errors jump out at me.

k,j,l = 4
y = numpy.arange(4).reshape(1,j)
z = numpy.arange(64).reshape(k,j,l)

y.dot(z) will combine the last dim of y with the 2nd to last of z ( j ) (see its doc). The result will be (1,k,l) .

np.einsum helps confirm this:

print y.dot(z)
print np.einsum('ij,kjl',y,z) # note the repeated j

both produce

array([[[ 56,  62,  68,  74],
        [152, 158, 164, 170],
        [248, 254, 260, 266],
        [344, 350, 356, 362]]])

yT (transpose) is (j,1). So the 2nd dot combines (1,k,l) with (j,1) , last dim ( l ) with first j , resulting in (1,k,1) , your 3d array:

array([[[ 420],
        [ 996],
        [1572],
        [2148]]])

np.einsum('ijl,lm', np.einsum('ij,kjl',y,z), y.T) # note the repeated l

This einsum can be combined into one call (using the same indexes):

np.einsum('ij,kjl,ml',y,z,y)

No broadcasting is involved. That is, no dimensions are being added or expanded. You just need to systematically track the dimensions as they pass through dot .

To get a smaller dimensional result, you need to squeeze out the 1's or reshape the result. If you start with a 1d y you get a 1d result

y1=y.squeeze()
np.einsum('j,kjl,l',y1,z,y1)
y1.dot(z).dot(y1.T)
# array([ 420,  996, 1572, 2148])