在两个最短的时间内找出两个1d阵列之间的差异，以形成一个二维阵列

Question

I am trying to find the absolute difference between every element of one array and every element of another to form a matrix.

I have achieved this using for loops but it is slow and I need it to be faster. I can do it faster in R for example by using the dist method but I am struggling to make it fast in C#.

double[] array1 = new double [] { 1.1, 2.0, 3.0, 4.0, 5.0 };
double[] array2 = new double[] { 6.1, 7.0, 8.0};    
double[,] final_array = new double[5, 3];
for (int i = 0; i < 5; i++)
{
    for (j = 0; j < 3; j++)
    {
        final_array[i,j] = Math.Abs(array1[i] - array2[j])
    }
}

# expected result of final_array
5    4.1    3.1     2.1     1.1
5.9  5      4       3       2
6.9  6      5       4       3

Although this result is the correct answer I want to do this faster as I will need to do this calculation for arrays of up to 15,000 in size.

Answer 1

You can use vectors in the System.Numerics namespace. The caveat is that it will only work with float , not with double . That shouldn't be a problem for subtraction though:

float[] array1 = new float[] { 1.1F, 2.0F, 3.0F, 4.0F, 5.0F };
float[] array2 = new float[] { 6.1F, 7.0F, 8.0F };    
float[,] final_array = new float[array1.Length, array2.Length];

int vectorCount = array2.Length / 4;
Vector4[] array2Vectors = new Vector4[vectorCount];
Parallel.For(0, vectorCount, i =>
{
    int offset = i * 4;
    array2Vectors[i] = new Vector4(array2[offset], array2[offset + 1],
        array2[offset + 2], array2[offset + 3]);
});

Parallel.For(0, array1.Length, i =>
{
    Vector4 v1 = new Vector4(array1[i], array1[i], array1[i], array1[i]);
    for (int j = 0; j < array2Vectors.Length; j++)
    {
        Vector4 result = Vector4.Abs(Vector4.Subtract(v1, array2Vectors[j]));
        int offset = j * 4;
        final_array[i, offset] = result.X;
        final_array[i, offset + 1] = result.Y;
        final_array[i, offset + 2] = result.Z;
        final_array[i, offset + 3] = result.W;
    }

    for (int j = vectorCount * 4; j < array2.Length; j++)
    {
        final_array[i,j] = Math.Abs(array1[i] - array2[j]);
    }
});

Since you are using vectors now, you will make use of the CPU's SIMD instructions, which should speed up your task.

Additional performance gains come from parallel execution with Parallel.For , which makes use of all available CPU cores.

You can try it out here.

Answer 2

There is no way to do this faster in terms of algorithmic complexity. It requires exactly O(n * m) operations to calculate this result, at least because you have the resulting array of that size.

There are some ways to slightly improve the performance of the code itself.
The easiest one is to switch to jagged arrays, as already suggested in the comments:

double[] array1 = new double [] { 1.1, 2.0, 3.0, 4.0, 5.0 };
double[] array2 = new double[] { 6.1, 7.0, 8.0};    
double[][] final_array = new double[5][];

for (int i = 0; i < 5; i++)
{
    final_array[i] = new double[3];
    for (int j = 0; j < 3; j++)
    {
        final_array[i][j] = Math.Abs(array1[i] - array2[j]);
    }
}

You can read more about multidimensional array vs jagged arrays and their performance here:
What are the differences between a multidimensional array and an array of arrays in C#?

You could also go further and increase performance by using unsafe pointers to access multidimensional array or by utilizing advanced processor instructions (intrinsics), but... the question is: is this really something you need to think of? Is it an only bottle neck in an extremely high-load system? If it is not, then just leave your code as it is, in a clearly readable and understandable form. Saying about performance, O(n * m) asymptotic complexity is perfectly fine for arrays of size 15000.