How to find the largest eigenvector of a matrix?

Question

I am trying to implement this without success and I have to do this without using external modules numpy, etc. There are 3 modules in the app I am coding this, Python and C#, C++ but no other fancy libraries other than standard ones.

On a separate application, I used numpy's svd and it works very accurately. So I am using it to match my results. My method is PCA and everything is good up to this point. But after I calculate my symmetric covariance matrix, I don't know how to find the largest eigenvector.

The data set is always 3d points, x, y and z.

vector center;
for(point p in points):
    center += p;
center /= points.count;

sumxx = 0;
sumxy = 0;
sumxz = 0;
sumyy = 0;
sumyz = 0;
sumzz = 0;
for(point p in points):
    vector s = p - center;
    sumxx += s.x * s.x;
    sumxy += s.x * s.y;
    sumxz += s.x * s.z;
    sumyy += s.y * s.y;
    sumyz += s.y * s.z;
    sumzz += s.z * s.z;

matrix3 mat = invert(matrix3(sumxx, sumxy, sumxz, sumxy, sumyy, sumyz, sumxz, sumyz, sumzz));
vector n;
if (determinant(mat) > 0)
    normal = find_largest_eigenvalue

Answer 1

Let us recap what you are asking, to clarify :

1. Find an eigenvector of a matrix mat
2. This eigenvector should be associated with the largest eigenvalue of the matrix
3. The matrix is the symmetric covariance matrix of a principal component analysis . In particular, it is symmetric.
4. Your matrix is square of size 3 by 3 , as shown in your code by matrix3 mat = ... and confirmed in a (now deleted) comment.

Under these very specific circumstances, the following answer applies. However tmyklebu warns against numerical instability of this approach for some pathological matrices, typically when r is close to -1 .

---

Alright, lets start with a bit of reading from wikipedia's page on Characteristic polynomials

In linear algebra, the characteristic polynomial of a square matrix is a polynomial, which is invariant under matrix similarity and has the eigenvalues as roots.

blah blah blah, let's skip directly to the 3x3 matrix section in the page on Eigenvalue algorithms .

If A is a 3×3 matrix, then its characteristic equation can be expressed as:

Followed a few lines later by (more or less) this pseudo-code, for symmetric matrices (which you say you have, if I'm not mistaken -- otherwise you might have complex eigenvalues) :

p1 = A(1,2)^2 + A(1,3)^2 + A(2,3)^2
if (p1 == 0) 
   % A is diagonal.
   eig1 = A(1,1)
   eig2 = A(2,2)
   eig3 = A(3,3)
else
   q = (A(1,1) + A(2,2) + A(3,3)) / 3
   p2 = (A(1,1) - q)^2 + (A(2,2) - q)^2 + (A(3,3) - q)^2 + 2 * p1
   p = sqrt(p2 / 6)
   B = (1 / p) * (A - q * I)       % I is the identity matrix
   r = determinant(B) / 2

   % In exact arithmetic for a symmetric matrix  -1 <= r <= 1
   % but computation error can leave it slightly outside this range.
   if (r <= -1) 
      phi = pi / 3
   elseif (r >= 1)
      phi = 0
   else
      phi = acos(r) / 3
   end

   % the eigenvalues satisfy eig3 <= eig2 <= eig1
   eig1 = q + 2 * p * cos(phi)
   eig3 = q + 2 * p * cos(phi + (2*pi/3))
   eig2 = 3 * q - eig1 - eig3     % since trace(A) = eig1 + eig2 + eig3
end

So you want max(eig1,eig2,eig3) in the first case, eig1 in the second case. Let us call e this largest eigenvalue.

For the eigenvector, you can now just solve (Ae*I)x=0

Answer 2

There are different algorithms for finding eigenvalues. Some work from smallest to largest, like QR; others work from largest to smallest, like power iteration or Jacobi-Davidson.

Maybe a switch of algorithm is what you want. Try power method and see if that helps.

https://scicomp.stackexchange.com/questions/1681/what-is-the-fastest-way-to-calculate-the-largest-eigenvalue-of-a-general-matrix