C# run method multiple time concurrently

Question

I have the following C# code:

string[] files = Directory.GetFiles(@"C:\Users\Documents\DailyData", "*.txt", SearchOption.AllDirectories);

        for (int ii = 0; ii < files.Length; ii++)
        {
            perFile(files[ii]);
        }

Where per file opens each file and creates a series of generic lists, then writes the results to a file.

Each iteration takes about 15 minutes.

I want the perFile method to execute concurrently eg not wait for the current iteration to finish before starting the next one.

My questions are:

How would I do this?

How can I control how many instances of perFile are running concurrently

How can I determine how many concurrent instances of perFile my computer can handle at one time

Answer 1

Try to use Parallel.ForEach . You can use ParallelOptions.MaxDegreeOfParallelism property to set max count of running perFile .

Answer 2

You can use Parallel.For to do what you need if you're using .Net 4.5 and above, You can control how many threads to use by providing a ParallelOptions with MaxDegreeOfParalleism set to a number you like.

Example code:

ParallelOptions POptions = new ParallelOptions();
POptions.MaxDegreeOfParalleism = ReplaceThisWithSomeNumber;

Parallel.For(0,files.Length,POptions,(x) => {perFile(files[x]);});

Note that your program will be IO bound though (read Asynchronous IO for more information on how to tackle that)