矩阵乘法性能

Question

Code goes as follows,

In [180]: rng = np.random.RandomState(123)

In [181]: A1 = rng.uniform(size=(10000,80))

In [182]: B1 = rng.uniform(size=(10000,30))

In [183]: A2 = rng.uniform(size=(80,10000))

In [184]: B2 = rng.uniform(size=(30,10000))

In [185]: %timeit np.dot(A1.T, B1)
10 loops, best of 3: 136 ms per loop

In [186]: %timeit np.dot(A2, B2.T)
10 loops, best of 3: 25.1 ms per loop

In [4]: %timeit np.dot(A2, B1)
10 loops, best of 3: 56.3 ms per loop

I want to multiply (A1, B1) and (A2, B2) to form a (80,30) matrix, the difference here is that A1 is defined as the transpose of A2 , with 10000 rows in A1 but 80 row in A2 . Same for B1,B2 .

The performance is quite different, I guess it's because the memory layout of numpy.array is more cache-friendly with large columns than with large rows , right? But how?

Answer 1

MCM is an algorithm that computes the most efficient way to multiply a set of matrices, it's worth studying to learn more about matrix multiplication. In general matrix multiplication is not commutative and depending on the order you can get radically different run times for computation.

Standard Wiki