Download Multiple Files From FTP Server Quickly

Question

I would like to download many (thousands) of smaller files from an FTP server using C#. With my current code I am not able to achieve speeds over 100 KB/s (usually much slower) (I am testing on a local FileZilla FTP server).

This is my code:

foreach (var file in files)
{
    //Client is basically a WebClient
    var stream = Client.OpenRead(new Uri(_serverRootPath + file.Replace(@"\", "/")));

    var filePath = _clientRootPath + file;
    if (!Directory.Exists(Path.GetDirectoryName(filePath)))
        Directory.CreateDirectory(Path.GetDirectoryName(filePath));
    var fileStream = new FileStream(_clientRootPath + file, FileMode.Create);

    const int bufferSize = 8192;
    var buffer = new byte[bufferSize];
    var readCount = stream.Read(new byte[bufferSize], 0, bufferSize);

    while (readCount > 0)
    {
    await fileStream.WriteAsync(buffer, 0, readCount);
    readCount = await stream.ReadAsync(buffer, 0, bufferSize);
    }

    stream.Close();
}

Any help would be appreciated.

Answer 1

Have you tried running this in parallel? If the speed of your connection is the bottleneck here then it won't help, otherwise its worth trying. Check http://msdn.microsoft.com/en-us/library/dd460720(v=vs.110).aspx for examples.