C# Multi-threading - Upload to FTP Server

Question

I would like to seek your help in implementing Multi-Threading in my C# program.

The program aims to upload 10,000++ files to an ftp server. I am planning to implement atleast a minimum of 10 threads to increase the speed of the process.

With this, this is the line of code that I have:

I have initialized 10 Threads:

public ThreadStart[] threadstart = new ThreadStart[10];
public Thread[] thread = new Thread[10];

My plan is to assign one file to a thread, as follows:

file 1 > thread 1
file 2 > thread 2
file 3 > thread 3
.
.
.
file 10 > thread 10
file 11 > thread 1
.
.
.

And so I have the following:

foreach (string file in files)
{
     loop++;

     threadstart[loop] = new ThreadStart(() => ftp.uploadToFTP(uploadPath + @"/" + Path.GetFileName(file), file));
     thread[loop] = new Thread(threadstart[loop]);
     thread[loop].Start();

     if (loop == 9)
     {
         loop = 0;
     }                          
}

The passing of files to their respective threads is working. My problem is that the starting of the thread is overlapping.

One example of exception is that when Thread 1 is running, then a file is passed to it. It returns an error since Thread 1 is not yet successfully done, then a new parameter is being passed to it. Also true with other threads.

What is the best way to implement this?

Any feedback will be greatly appreciated. Thank you! :)

Answer 1

Using async-await and just pass an array of files into it:

private static async void TestFtpAsync(string userName, string password, string ftpBaseUri,
      IEnumerable<string> fileNames)
    {
      var tasks = new List<Task<byte[]>>();
      foreach (var fileInfo in fileNames.Select(fileName => new FileInfo(fileName)))
      {
        using (var webClient = new WebClient())
        {
          webClient.Credentials = new NetworkCredential(userName, password);
          tasks.Add(webClient.UploadFileTaskAsync(ftpBaseUri + fileInfo.Name, fileInfo.FullName));
        }
      }
      Console.WriteLine("Uploading...");
      foreach (var task in tasks)
      {
        try
        {
          await task;
          Console.WriteLine("Success");
        }
        catch (Exception ex)
        {
          Console.WriteLine(ex.ToString());
        }
      }
    }

Then call it like this:

  const string userName = "username";
  const string password = "password";
  const string ftpBaseUri = "ftp://192.168.1.1/";
  var fileNames = new[] { @"d:\file0.txt", @"d:\file1.txt", @"d:\file2.txt" };
  TestFtpAsync(userName, password, ftpBaseUri, fileNames);

Answer 2

Why doing it the hard way? .net already has a class called ThreadPool. You can just use that and it manages the threads itself. Your code will be like this:

 static void DoSomething(object n)
    {
        Console.WriteLine(n);
        Thread.Sleep(10);
    }

    static void Main(string[] args)
    {
        ThreadPool.SetMaxThreads(20, 10);
        for (int x = 0; x < 30; x++)
        {
            ThreadPool.QueueUserWorkItem(new WaitCallback(DoSomething), x);
        }
        Console.Read();
    }