Multiple thread accessing and editing the same double array

Question

I need to iterate through every double in an array to do the "Laplacian Smoothing", "mixing values" with neighbour doubles.

I'll keep stored values in a temp clone array update the original at the end.

Pseudo code:

double[] A = new double[1000];
// Filling A with values...

double[] B = A.Clone as double[];

for(int loops=0;loops<10;loops++){ // start of the loop

    for(int i=0;i<1000;i++){ // iterating through all doubles in the array
    // Parallel.For(0, 1000, (i) => {

       double v= A[i];
       B[i]-=v;
       B[i+1]+=v/2;
       B[i-1]+=v/2;
       // here i'm going out of array bounds, i know. Pseudo code, not relevant.

    }
    // });
}
A = B.Clone as double[];

With for it works correctly. "Smoothing" the values in the array.

With Parallel.For() I have some access sync problems: threads are colliding and some values are actually not stored correctly. Threads access and edit the array at the same index many times.

(I haven't tested this in a linear array, i'm actually working on a multidimensional array[x,y,z]..)

How can I solve this?

I was thinking to make a separate array for each thread, and do the sum later... but I need to know the thread index and I haven't found anywhere in the web. (I'm still interested if a "thread index" exist even with a totally different solution...).

I'll accept any solution.

Answer 1

You probably need one of the more advanced overloads of the Parallel.For method:

public static ParallelLoopResult For<TLocal>(int fromInclusive, int toExclusive,
    ParallelOptions parallelOptions, Func<TLocal> localInit,
    Func<int, ParallelLoopState, TLocal, TLocal> body,
    Action<TLocal> localFinally);

Executes a for loop with thread-local data in which iterations may run in parallel, loop options can be configured, and the state of the loop can be monitored and manipulated.

This looks quite intimidating with all the various lambdas it expects. The idea is to have each thread work with local data, and finally merge the data at the end. Here is how you could use this method to solve your problem:

double[] A = new double[1000];
double[] B = (double[])A.Clone();
object locker = new object();
var parallelOptions = new ParallelOptions()
{
    MaxDegreeOfParallelism = Environment.ProcessorCount
};
Parallel.For(0, A.Length, parallelOptions,
    localInit: () => new double[A.Length], // create temp array per thread
    body: (i, state, temp) =>
    {
        double v = A[i];
        temp[i] -= v;
        temp[i + 1] += v / 2;
        temp[i - 1] += v / 2;
        return temp; // return a reference to the same temp array
    }, localFinally: (localB) =>
    {
        // Can be called in parallel with other threads, so we need to lock
        lock (locker)
        {
            for (int i = 0; i < localB.Length; i++)
            {
                B[i] += localB[i];
            }
        }
    });

I should mention that the workload of the above example is too granular, so I wouldn't expect large improvements in performance from the parallelization. Hopefully your actual workload is more chunky. If for example you have two nested loops, parallelizing only the outer loop will work greatly because the inner loop will provide the much needed chunkiness.

---

Alternative solution: Instead of creating auxiliary arrays per thread, you could just update directly the B array, and use locks only when processing an index in the dangerous zone near the boundaries of the partitions :

Parallel.ForEach(Partitioner.Create(0, A.Length), parallelOptions, range =>
{
    bool lockTaken = false;
    try
    {
        for (int i = range.Item1; i < range.Item2; i++)
        {
            bool shouldLock = i < range.Item1 + 1 || i >= range.Item2 - 1;
            if (shouldLock) Monitor.Enter(locker, ref lockTaken);
            double v = A[i];
            B[i] -= v;
            B[i + 1] += v / 2;
            B[i - 1] += v / 2;
            if (shouldLock) { Monitor.Exit(locker); lockTaken = false; }
        }
    }
    finally
    {
        if (lockTaken) Monitor.Exit(locker);
    }
});

Answer 2

I need to iterate through every double in an array to do the "Laplacian Smoothing", "mixing values" with neighbour doubles.

I'll keep stored values in a temp clone array update the original at the end.

Pseudo code:

double[] A = new double[1000];
// Filling A with values...

double[] B = A.Clone as double[];

for(int loops=0;loops<10;loops++){ // start of the loop

    for(int i=0;i<1000;i++){ // iterating through all doubles in the array
    // Parallel.For(0, 1000, (i) => {

       double v= A[i];
       B[i]-=v;
       B[i+1]+=v/2;
       B[i-1]+=v/2;
       // here i'm going out of array bounds, i know. Pseudo code, not relevant.

    }
    // });
}
A = B.Clone as double[];

With for it works correctly. "Smoothing" the values in the array.

With Parallel.For() I have some access sync problems: threads are colliding and some values are actually not stored correctly. Threads access and edit the array at the same index many times.

(I haven't tested this in a linear array, i'm actually working on a multidimensional array[x,y,z]..)

How can I solve this?

I was thinking to make a separate array for each thread, and do the sum later... but I need to know the thread index and I haven't found anywhere in the web. (I'm still interested if a "thread index" exist even with a totally different solution...).

I'll accept any solution.

Answer 3

Ok, it appears that modulus can solve pretty much all my problems. Here a really simplified version of the working code: (the big script is 3d and unfinished... )

private void RunScript(bool Go, ref object Results)
  {
    if(Go){
      LaplacianSmooth(100);
      // Needed to restart "RunScript" over and over
      this.Component.ExpireSolution(true);
    }
    else{
      A = new double[count];
      A[100] = 10000;
      A[500] = 10000;
    }
    Results = A;
  }

  // <Custom additional code> 
  public static int T = Environment.ProcessorCount;
  public static int count = 1000;
  public double[] A = new double[count];
  public double[,] B = new double[count, T];

  public void LaplacianSmooth(int loops){
    for(int loop = 0;loop < loops;loop++){

      B = new double[count, T];

      // Copying values to first column of temp multidimensional-array
      Parallel.For(0, count, new ParallelOptions { MaxDegreeOfParallelism = T }, i => {
        B[i, 0] = A[i];
        });

      // Applying Laplacian smoothing
      Parallel.For(0, count, new ParallelOptions { MaxDegreeOfParallelism = T }, i => {
        int t = i % 16;
        // Wrapped next and previous element indexes
        int n = (i + 1) % count;
        int p = (i + count - 1) % count;
        double v = A[i] * 0.5;
        B[i, t] -= v;
        B[p, t] += v / 2;
        B[n, t] += v / 2;
        });

      // Copying values back to main array
      Parallel.For(0, count, new ParallelOptions { MaxDegreeOfParallelism = T }, i => {
        double val = 0;
        for(int t = 0;t < T;t++){
          val += B[i, t];
        }
        A[i] = val;
        });
    }
  }

There are no "collisions" with the threads, as confirmed by the result of "Mass Addition" (a sum) that is constant at 20000.

Thanks everyone for the tips!