how are numpy eigenvalues and eigenvectors computed

Question

I am using numpy for calculating eigenvalues and eigenvectors of a symmetrical, square array. My array is:

L = [[ 2. -1. -1.  0.  0.  0.]
     [-1.  3.  0. -1.  0. -1.]
     [-1.  0.  2. -1.  0.  0.]
     [ 0. -1. -1.  3. -1.  0.]
     [ 0.  0.  0. -1.  2. -1.]
     [ 0. -1.  0.  0. -1.  2.]]

The results when executing numpy.linalg.eig(L) are show below

eigenvalues:

[ 5.00000000e+00,   
  3.96872205e-16,   
  1.00000000e+00,
  2.00000000e+00,   
  3.00000000e+00,   
  3.00000000e+00 ]

eigenvectors:

[[ -2.88675135e-01   4.08248290e-01  -5.00000000e-01  4.08248290e-01   -4.36632863e-01   4.44614891e-01]
 [  5.77350269e-01   4.08248290e-01  -3.34129212e-16  4.08248290e-01   -1.08813217e-01  -5.41271705e-01]
 [  2.88675135e-01   4.08248290e-01  -5.00000000e-01  4.08248290e-01    5.45446080e-01   9.66568140e-02]
 [ -5.77350269e-01   4.08248290e-01   1.06732810e-16  4.08248290e-01   -1.08813217e-01  -5.41271705e-01]
 [  2.88675135e-01   4.08248290e-01   5.00000000e-01  4.08248290e-01   -4.36632863e-01   4.44614891e-01]
 [ -2.88675135e-01   4.08248290e-01   5.00000000e-01 -4.08248290e-01    5.45446080e-01   9.66568140e-02]]

The results are close (if normalized) to those you get when you analytically compute them, but some errors seem to introduce in both eigenvalues and eigenvectors. Is there some way to bypass these errors using numpy?

Where are these errors come from? What algorithm numpy uses?

Answer 1

If you want the precision of the analytic derivation, you will need to use symbolic computation , which is what Wolfram Alpha, Mathematica, and related systems use. In Python, you may want to look into SymPy , for example.

The numerical computation that is embedded into the NumPy package you're using is inherently subject to the small errors and vicissitudes of floating point numerical representations . Such errors and approximations are unavoidable with numerical computing.

Here's an example:

from sympy import Matrix, pretty

L = Matrix([[ 2, -1, -1,  0,  0,  0,],
     [-1,  3,  0, -1,  0, -1,],
     [-1,  0,  2, -1,  0,  0,],
     [ 0, -1, -1,  3, -1,  0,],
     [ 0,  0,  0, -1,  2, -1,],
     [ 0, -1,  0,  0, -1,  2,]])

print "eigenvalues:"
print pretty(L.eigenvals())
print
print "eigenvectors:"
print pretty(L.eigenvects(), num_columns=132)

Yields:

eigenvalues:
{0: 1, 1: 1, 2: 1, 3: 2, 5: 1}

eigenvectors:
⎡⎛0, 1, ⎡⎡1⎤⎤⎞, ⎛1, 1, ⎡⎡-1⎤⎤⎞, ⎛2, 1, ⎡⎡1 ⎤⎤⎞, ⎛3, 2, ⎡⎡1 ⎤, ⎡0 ⎤⎤⎞, ⎛5, 1, ⎡⎡1 ⎤⎤⎞⎤
⎢⎜      ⎢⎢ ⎥⎥⎟  ⎜      ⎢⎢  ⎥⎥⎟  ⎜      ⎢⎢  ⎥⎥⎟  ⎜      ⎢⎢  ⎥  ⎢  ⎥⎥⎟  ⎜      ⎢⎢  ⎥⎥⎟⎥
⎢⎜      ⎢⎢1⎥⎥⎟  ⎜      ⎢⎢0 ⎥⎥⎟  ⎜      ⎢⎢1 ⎥⎥⎟  ⎜      ⎢⎢-1⎥  ⎢-1⎥⎥⎟  ⎜      ⎢⎢-2⎥⎥⎟⎥
⎢⎜      ⎢⎢ ⎥⎥⎟  ⎜      ⎢⎢  ⎥⎥⎟  ⎜      ⎢⎢  ⎥⎥⎟  ⎜      ⎢⎢  ⎥  ⎢  ⎥⎥⎟  ⎜      ⎢⎢  ⎥⎥⎟⎥
⎢⎜      ⎢⎢1⎥⎥⎟  ⎜      ⎢⎢-1⎥⎥⎟  ⎜      ⎢⎢-1⎥⎥⎟  ⎜      ⎢⎢0 ⎥  ⎢1 ⎥⎥⎟  ⎜      ⎢⎢-1⎥⎥⎟⎥
⎢⎜      ⎢⎢ ⎥⎥⎟  ⎜      ⎢⎢  ⎥⎥⎟  ⎜      ⎢⎢  ⎥⎥⎟  ⎜      ⎢⎢  ⎥  ⎢  ⎥⎥⎟  ⎜      ⎢⎢  ⎥⎥⎟⎥
⎢⎜      ⎢⎢1⎥⎥⎟  ⎜      ⎢⎢0 ⎥⎥⎟  ⎜      ⎢⎢-1⎥⎥⎟  ⎜      ⎢⎢-1⎥  ⎢-1⎥⎥⎟  ⎜      ⎢⎢2 ⎥⎥⎟⎥
⎢⎜      ⎢⎢ ⎥⎥⎟  ⎜      ⎢⎢  ⎥⎥⎟  ⎜      ⎢⎢  ⎥⎥⎟  ⎜      ⎢⎢  ⎥  ⎢  ⎥⎥⎟  ⎜      ⎢⎢  ⎥⎥⎟⎥
⎢⎜      ⎢⎢1⎥⎥⎟  ⎜      ⎢⎢1 ⎥⎥⎟  ⎜      ⎢⎢-1⎥⎥⎟  ⎜      ⎢⎢1 ⎥  ⎢0 ⎥⎥⎟  ⎜      ⎢⎢-1⎥⎥⎟⎥
⎢⎜      ⎢⎢ ⎥⎥⎟  ⎜      ⎢⎢  ⎥⎥⎟  ⎜      ⎢⎢  ⎥⎥⎟  ⎜      ⎢⎢  ⎥  ⎢  ⎥⎥⎟  ⎜      ⎢⎢  ⎥⎥⎟⎥
⎣⎝      ⎣⎣1⎦⎦⎠  ⎝      ⎣⎣1 ⎦⎦⎠  ⎝      ⎣⎣1 ⎦⎦⎠  ⎝      ⎣⎣0 ⎦  ⎣1 ⎦⎦⎠  ⎝      ⎣⎣1 ⎦⎦⎠⎦

While the ASCII pretty-printer is, um, working hard to provide even quasi-good looking output, you can see that you are getting symbolically computed, precise output. If you're using IPython and have it set up to show LaTeX output, you'll get a nicer display .

Answer 2

It looks like it is using an iterative method from LAPACK. It converges to a solution. If it doesn't converge, it throw an exception.

Since you know the matrix is symmetric, you may do better with eigh. http://docs.scipy.org/doc/numpy/reference/generated/numpy.linalg.eigh.html

Documentation Page: http://docs.scipy.org/doc/numpy/reference/generated/numpy.linalg.eig.html

Source Code: https://github.com/numpy/numpy/blob/v1.9.1/numpy/linalg/linalg.py#L982