C＃-将数据从ThreadPool线程传递回主线程

Question

Current implementation: Waits until parallelCount values are collected, uses ThreadPool to process the values, waits until all threads complete, re-collect another set of values and so on...

Code:

private static int parallelCount = 5;
private int taskIndex;
private object[] paramObjects;

// Each ThreadPool thread should access only one item of the array, 
// release object when done, to be used by another thread
private object[] reusableObjects = new object[parallelCount];     

private void MultiThreadedGenerate(object paramObject)
{
    paramObjects[taskIndex] = paramObject;
    taskIndex++;

    if (taskIndex == parallelCount)
    { 
        MultiThreadedGenerate();

        // Reset
        taskIndex = 0;
    }
}

/*
 * Called when 'paramObjects' array gets filled
 */
private void MultiThreadedGenerate()
{
    int remainingToGenerate = paramObjects.Count;

    resetEvent.Reset();

    for (int i = 0; i < paramObjects.Count; i++)
    {
        ThreadPool.QueueUserWorkItem(delegate(object obj)
        {
            try
            {
                int currentIndex = (int) obj;       

                Generate(currentIndex, paramObjects[currentIndex], reusableObjects[currentIndex]);
            }
            finally
            {
                if (Interlocked.Decrement(ref remainingToGenerate) == 0)
                {
                    resetEvent.Set();
                }
            }
        }, i);
    }

    resetEvent.WaitOne();    
}

I've seen significant performance improvements with this approach, however there are a number of issues to consider:

[1] Collecting values in paramObjects and synchronization using resetEvent can be avoided as there is no dependency between the threads (or current set of values with the next set of values). I'm only doing this to manage access to reusableObjects (when a set paramObjects is done processing, I know that all objects in reusableObjects are free, so taskIndex is reset and each new task of the next set of values will have its unique 'reusableObj' to work with).

[2] There is no real connection between the size of reusableObjects and the number of threads the ThreadPool uses. I might initialize reusableObjects to have 10 objects, and say due to some limitations, ThreadPool can run only 3 threads for my MultiThreadedGenerate() method, then I'm wasting memory.

So by getting rid of paramObjects , how can the above code be refined in a way that as soon as one thread completes its job, that thread returns its taskIndex (or the reusableObj ) it used and no longer needs so that it becomes available to the next value. Also, the code should create a reUsableObject and add it to some collection only when there is a demand for it. Is using a Queue here a good idea ?

Thank you.

Answer 1

There's really no reason to do your own manual threading and task management any more. You could restructure this to a more loosely-coupled model using Task Parallel Library (and possibly System.Collections.Concurrent for result collation).

Performance could be further improved if you don't need to wait for a full complement of work before handing off each Task for processing.

TPL came along in .Net 4.0 but was back-ported to .Net 3.5 . Download here .