Matrix multiplication in C#

Question

I'm having problems multiplying two matrices of dimensions 5000x1024. I tried to do it through loops in a conventional manner, but it takes forever. Is there any good library with implemented and optimized matrix operations, or any algorithm that does it without 3 loops?

Answer 1

Have you looked into using OpenCL? One of the examples in the Cloo (C# OpenCL library) distribution is a large 2D matrix multiplication.

Unlike CUDA, OpenCL kernels will run on your GPU (if available and supported) or the CPU. On the GPU you'll see really, really, really dramatic speed increases. I mean, really dramatic, on the order of 10x-100x depending on how efficient your kernel is and how many cores your GPU has. (A Fermi-based NVidia card will have between 384-512, and the new 600's have something like 1500.)

If you're not interested in going that route - though anyone that's doing numerically-intensive, easily parallelizable operations like this should be using the GPU - make sure you're at least using C#'s built-in parallelization:

Parallel.For(
  0
  ,5000
  , (i) => { 
    for(var j=0;j<1024;j++)
    {
      result[i,j] = .....
    }
);

also, check out GPU.NET and Brahma. Brahma lets you build OpenCL kernels in C# with LINQ. Definitely lowers the learning curve.

Answer 2

Take a look at Strassen algorithm which has a runtime of approx. O(n ^2.8 ) instead of O(n ³ ) with a naive method of multiplying matrices. One problem is that is not always stable but works fine for really high dimensions. Furthermore it is really complex so I would suggest you to rethink your design and maybe decrease the size of the matrix or split your problem into smaller pieces.

But keep in mind that a matrix multiplication with no special properties (like Aidan mentioned) is nearly impossible to optimize. Here an example: the Coppersmith-Winograd algorithm takes O(n ^2.3737 ) and it is by far one of the best algorithms for matrix multiplication! The best option here would be to either use OpenCL and the GPU (mentioned by David ) or to look at other optimized programming languages like Python with the package numpy .

Good luck either way!