Using multiple threads in C#

Question

I want to do independent tasks of parsing multiple files in a system and get the version of each as follows:

public void obtainVersionList()
{

    for(int iterator = 1; iterator < list.length; iterator++) //list stores all the file names
    {
        Thread t = new Thread( () => GetVersion(ref list[iterator]) 
        //list will again store the fileVersions using GetVersion()
    }
}

Here,

1. I get Index out of bounds exception. How's that possible as I've checked a condition iterator < list.length. Is this due to multiple threads running?
2. How to minimize the operation time when we parse multiple files in the disk?

Answer 1

For parallel executions I'd recommend you Parallel.ForEach (or the Task class):

Parallel.ForEach(list, item => GetVersion(ref item));

The TPL you use then does the thread management for you, typically by using a thread pool. You can, however, use different scheduler implementations. In general, re-using threads is cheaper than spawning many.

Inspired by weston's suggestions I tried out an alternative, which may be considered creative LINQ usage :

static void Main(string[] args)
{
    var seq = Enumerable.Range(0, 10).ToList();
    var tasks = seq
        .Select(i => Task.Factory.StartNew(() => Foo(i)))
        .ToList(); // important, spawns the tasks
    var result = tasks.Select(t => t.Result);

    // no results are blockingly received before this
    // foreach loop
    foreach(var r in result)
    {
        Console.WriteLine(r);
    }
}

static int Foo(int i)
{
    return i;
}

For each input in seq I create a Task<T> doing something. The Result of these tasks is collected in result , which is not iterated before the foreach . This code does maintain the order of your results too.

The sample does not modify seq . This is a different concept than altering list as you want to do.

Answer 2

The iterator variable is being captured by reference, not by value. That makes all threads share the same variable. Copy it to a loop-local variable first before using it in the lambda.

Everyone falls for this at least once. The C# designers have regretted this decision so much they consider changing it.

Answer 3

To solve the index out of bounds problem you could make a local copy of the iteration variable:

for(int iterator = 1; iterator < list.length; iterator++) //list stores all the file names
{
     int iterator1 = iterator;
     Thread t = new Thread( () => GetVersion(ref list[iterator1]);
     //list will again store the fileVersions using GetVersion()
}

2) How to minimize the operation time when we parse multiple files in the disk?

That's not really a good idea when you have a single mechanical disk. You're only bouncing the mechanical head around as each thread gets a chance to run. Stick to a single thread for disk I/O.

Answer 4

See this question

Do not close over your iterator variable. Instead, create a local variable and close over that:

public void obtainVersionList()
{
    //list stores all the file names
    for(int iterator = 1; iterator < list.length; iterator++) 
    {
        //list will again store the fileVersions using GetVersion()
        var local = list[iterator];
        Thread t = new Thread( () => GetVersion(ref local);
    }
}

Answer 5

You shouldn't let multiple threads adjust the same list. This is not threadsafe unless the list is threadsafe. I don't know the type, but List<string> isn't.

The other thing is that you shouldn't create own threads for this. What if the list is 200 files, your PC will grind to a halt creating 200 threads. Let threadpool do the work of managing a sensible number of threads for you.

This solution assumes you have .net4.

Change signature of GetVersion to: private static string GetVersion(string file)

        var tasks = new List<Task>();
        //start tasks
        foreach (var file in list)
        {
            var localFile = file; //local variable on advice of resharper
            tasks.Add(Task<string>.Factory.StartNew(() => GetVersion(localFile)));
        }
        //wait for them to complete
        Task.WaitAll(tasks.ToArray());
        //read the results
        IEnumerable<string> result = tasks.OfType<Task<string>>().Select(e => e.Result);
        //print em out for test
        foreach (var str in result)
        {
            Console.WriteLine(str);
        }