How can I make this recursive function more efficient?

Question

I created this function to recursively copy an entire directory from an FTP server. It works just fine except that it is about 4 times slower than using FileZilla to do the same operation. It takes approximately 55 seconds to download the directory in FileZilla but it takes 229 seconds with this function. What can I do to make it download/run faster?

Private Sub CopyEntireDirectory(ByVal directory As String)
    Dim localPath = localDirectory & formatPath(directory)
    'creates directory in destination path
    IO.Directory.CreateDirectory(localPath)

    'Gets the directory details so I can separate folders from files
    Dim fileList As ArrayList = Ftp.ListDirectoryDetails(directory, "")

    For Each item In fileList
        'checks if it's a folder or file: d=folder
        If (item.ToString().StartsWith("d")) Then
            'gets the directory from the details
            Dim subDirectory As String = item.ToString().Substring(item.ToString().LastIndexOf(" ") + 1)
            CopyEntireDirectory(directory & "/" & subDirectory)
        Else
            Dim remoteFilePath As String = directory & "/" & item.ToString().Substring(item.ToString().LastIndexOf(" ") + 1)
            Dim destinationPath = localPath & "\" & item.ToString().Substring(item.ToString().LastIndexOf(" ") + 1)
            'downloads file to destination directory
            Ftp.DownLoadFile(remoteFilePath, destinationPath)
        End If
    Next
End Sub

Below is the download function that is taking up all the time.

Public Sub DownLoadFile(ByVal fromFilename As String, ByVal toFilename As String)
    Dim files As ArrayList = Me.ListDirectory(fromFilename, "")
    Dim request As FtpWebRequest = Me.CreateRequestObject(fromFilename)
    request.Method = WebRequestMethods.Ftp.DownloadFile

    Dim response As FtpWebResponse = CType(request.GetResponse(), FtpWebResponse)
    If response.StatusCode <> FtpStatusCode.OpeningData AndAlso response.StatusCode <> FtpStatusCode.DataAlreadyOpen Then
        Throw New ApplicationException(Me.BuildCustomFtpErrorMessage(request, response))
    End If

    Dim fromFilenameStream As Stream = response.GetResponseStream()
    Dim toFilenameStream As FileStream = File.Create(toFilename)

    Dim buffer(BLOCK_SIZE) As Byte
    Dim bytesRead As Integer = fromFilenameStream.Read(buffer, 0, buffer.Length)
    Do While bytesRead > 0
        toFilenameStream.Write(buffer, 0, bytesRead)
        Array.Clear(buffer, 0, buffer.Length)
        bytesRead = fromFilenameStream.Read(buffer, 0, buffer.Length)
    Loop

    response.Close()
    fromFilenameStream.Close()
    toFilenameStream.Close()
End Sub

Answer 1

The slowness would obviously be within then FTP commands. Running your other code recursively would likely be able to run a million times per second because there is nothing to it.

The FTP download (whatever it is) should have the ability to define the size of the chunks is grabs. This will be key in your speed. It needs to be optimized based on your connection speed and file size. There is no RIGHT number for everyone.

EDIT

Based on the new code, the issue is in your BLOCK_SIZE which I assume is a constant. Play with the size of this to get your optimal speed.

HINT: This should be a multiple of 1024