Faster Implementation of Matrix-Vector Multiplication [Parallel Computation]

Question

I have this code that performs a matrix-vector multiplication written in php.

Here's a snippet:

for($i = 0; $i < sizeof($transposed_matrix); $i++) {
            $vector[$i] = 0;
            for($j = 0; $j < sizeof($new_vector); $j++) {
                $vector[$i] += ($transposed_matrix[$i][$j] * $new_vector[$j]);
            }
        }

I want to know if there is any way to make this code run faster?

Answer 1

one optimization would be to count before for :

$size = sizeof($transposed_matrix);
$size2 = sizeof($new_vector);
for($i = 0; $i < $size; $i++) {
    $vector[$i] = 0;        
    for($j = 0; $j < $size2; $j++) {
        $vector[$i] += ($transposed_matrix[$i][$j] * $new_vector[$j]);
    }
}

Answer 2

PHP does not give you enough control for some serious optimization. The proposed improvements will probably have relatively small impact, unless you are multiplying huge matrices and vectors (in which case you should not be using PHP in the first place).

In addition to precomputing the sizes and using pre-increments for the counters (as suggested by Tjoene), use a temporary variable for the sum in the inner loop like so:

$sum = 0;
for ($j = 0; $j < $numCols; ++$j) {
    $sum += $matrix[$i][$j] * $vector[$j];
}
$vector[$i] = $sum;

This will avoid computing the correct destination location in $vector multiple times.

Probably the biggest performance gain can be achieved by storing the matrix data in a single flat array instead of the nested structure you use. Simply concatenate the rows of the matrix and you can run through its elements using a single index like so:

for ($i = 0, $n = 0; $i < $numRows; ++$i, ++$n)
{
    $sum = 0;
    for ($j = 0; $j < $numCols; ++$j) {
        $sum += $matrix[$n] * $vector[$j];
    }
    $vector[$i] = $sum;
}

This will, of course, only speed things up if you don't have to convert to this matrix layout prior to the actual multiplication.

If you don't want to change the matrix layout, you could speed things up by using foreach in the outer loop to retrieve the matrix's rows. Note, however, that this iterates over the set of rows in the order in which these row arrays were added to the matrix! If this order differs between the matrix and the vector, the result will be all wrong. So, probably not such a reliable thing to do as it breaks far to easily ...

Oh, an you can always try to partially unroll the loop(s).

Answer 3

PHP arrays have a tendency to be slow, this is a tribute to the hashing mechanism. PHP array performance . If you have a way to predetermine the size of the vectors, you could unroll the loops and avoid using Arrays. If your code is the entire code, this won't help you though, as every item in $transposed_matrix is hit only once, and you can reduce the number of hits on $vector by using the $sum technique as outlined by Atze Kaputnik. So you'll end up copying stuff from the array parameters to local variables, then calculate and then copy back... that'll more than kill the performance gain.

In the end, all you can do is switch to an entirely different method of optimization: JIT-compilers like HipHop or compiled languages. The same loop in C is likely to run in the order of 10 to 100 times faster, minus the time to fork that process.