How to calculate large-scale inverse matrix in python

Question

As we know that we can calculate the inverse matrix with the help of numpy as the following.

matrix1 = np.matrix([[8,2,5],[7,3,1],[4,9,6]])
inverse_matrix1 = matrix1.I
result = np.matmul(matrix1, inverse_matrix1)

The result is as the following and it is easy for us to check the accuracy simply by doing np.matmul.

matrix([[ 0.03585657,  0.1314741 , -0.05179283],
    [-0.15139442,  0.11155378,  0.10756972],
    [ 0.20318725, -0.25498008,  0.03984064]])

The checked result is as the following.

matrix([[ 1.00000000e+00,  0.00000000e+00,  2.77555756e-17],
    [ 0.00000000e+00,  1.00000000e+00,  3.46944695e-17],
    [-2.22044605e-16,  0.00000000e+00,  1.00000000e+00]])

However, this case is very small. In practice, although we should avoid calculating the inverse matrix for the large matrix, sometimes we have to do that. I found that matrix.I cannot provide me a relatively accurate inverse matrix when the matrix is relatively large. The example is shown as the following. I want to calculate the inverse matrix of a Gaussian Kernel Matrix whose shape is (300, 300).

point = np.reshape(np.linspace(-5.0, 5.0, 300), (300, 1))
kernel_matrix_np = np.exp(-(point - np.transpose(point))**2 / (2 * 2**2))

I am not sure how I can calculate such matrix. Thank you so much!

Answer 1

The difference in results between your examples isn't due to the size of the matrices, it's due to the rank. The first matrix is full rank

>>> matrix_rank(matrix1)
3  ## Shape of the matrix

the shape of the matrix), while in the second case the matrix is of rank 19.

>>> matrix_rank(kernel_matrix_np)
19   ## Much less than the shape of the matrix

It isn't possible to recover the original matrix in this case - a full rank matrix is required. As @Brenlla mentioned, this is reflected in the condition number. Roughly speaking each order of magnitude in the condition number represents one digit of precision lost.

>>> cond(kernel_matrix_np)
1.9605027391309521e+19

These are just two things to check any time calculations are made using matrices, and generally one of these will indicate where a problem is. Lastly, there are some cases using pinv in place of inv gives slightly better results, although that won't work in this case since the matrix isn't full rank.