Matlab inverse of large matrix

Question

This is the equation I am trying to solve:

h = (X'*X)^-1*X'*y

where X is a matrix and y is a vector ((X'X)^-1 is the inverse of X-transpose times X). I have coded this in Matlab as:

h = (X'*X)\X'*y

which I believe is correct. The problem is that X is around 10000x10000, and trying to calculate that inverse is crashing Matlab on even the most powerful computer I can find (16 cores, 24GB RAM). Is there any way to split this up, or a library designed for doing such large inversions?

Thank you.

Answer 1

That looks like a pseudo inverse. Are you perhaps looking for just

h = X \ y;

Answer 2

I generated a random 10,000 by 10,000 matrix X and a random 10,000 by 1 vector y.

I just broke up my computation step by step. (Code shown below)

1. Computed the transpose and held it in matrix K
2. Then I computed Matrix A by multiplying K by X
3. Computed vector b by multiplying K by vector y
4. Lastly, I used the backslash operator on A and b to solve

I didn't have a problem with the computation. It took a while, but breaking up the operations into the smallest groups possible helped to prevent the computer from being overwhelmed. However, it could be the composition of the matrix that you are using (ie. Sparse, decimals, etc.).

X = randi(2000, [10000, 10000]);
y = randi(2000, 10000, 1);
K = X';
A = K*X;
b = K*y;
S = A\b;

Answer 3

If you have multiple machines at your disposal, and you can recast your problem into the form h = X\\y as proposed by @Ben, then you could use distributed arrays. This demo shows how you can do that.

Answer 4

Jordan, Your equation is exactly the definition for "Moore-Penrose Matrix Inverse".

Check: http://mathworld.wolfram.com/Moore-PenroseMatrixInverse.html

Directly using h = X \\ y; should help. Or check Matlab pinv(X)*y