How to download directories from FTP using VB.NET

Question

I am trying to download multiple directories from FTP server to my local machine,

I have tried this,

Const localFile As String = "C:\Documents and Settings\cr\Desktop\T\New Folder\"
Const remoteFile As String = "textbox.Text"
Const host As String = "ftp://ftp.example.com"
Const username As String = "username"
Const password As String = "password"

For i1 = 0 To ListBox1.SelectedItems.Count - 1
    Dim li As New ListViewItem
    li = ListView1.Items.Add(ListBox1.SelectedItems(i1))
    Dim URI1 As String = host + remoteFile & "/" & ListBox1.SelectedItems(i1)
    Dim ftp1 As System.Net.FtpWebRequest = CType(FtpWebRequest.Create(URI1), FtpWebRequest)
    ftp1.Credentials = New System.Net.NetworkCredential(username, password)
    ftp1.KeepAlive = False
    ftp1.UseBinary = True
    ftp1.Method = System.Net.WebRequestMethods.Ftp.DownloadFile
    Using response As System.Net.FtpWebResponse = CType(ftp1.GetResponse, System.Net.FtpWebResponse)
        Using responseStream As IO.Stream = response.GetResponseStream

            Dim length As Integer = response.ContentLength
            Dim bytes(length) As Byte

            'loop to read & write to file
            Using fs As New IO.FileStream(localFile & ListBox1.SelectedItems(i1), IO.FileMode.Create)
                Dim buffer(2047) As Byte
                Dim read As Integer = 1

                Do
                    read = responseStream.Read(buffer, 0, buffer.Length)
                    fs.Write(buffer, 0, read)

                Loop Until read = 0 'see Note(1)
                responseStream.Close()
                fs.Flush()
                fs.Close()
            End Using
            responseStream.Close()
        End Using

        response.Close()
    End Using
    li.BackColor = Color.Aquamarine
Next

But here the problem is that I am able to download multiple files from folders, but unable to download the sub directories and their contents from the main directory.

Basically the main directory consist of files and sub directories both. So is there any possible way to download sub directory and its contents from FTP?

Thanks in advance.

Answer 1

Translating my answer to C# Download all files and subdirectories through FTP to VB.NET:

The FtpWebRequest does not have any explicit support for recursive file download (or any other recursive operation). You have to implement the recursion yourself:

• List the remote directory
• Iterate the entries, downloading files and recursing into subdirectories (listing them again, etc.)

A tricky part is to identify files from subdirectories. There's no way to do that in a portable way with the FtpWebRequest . The FtpWebRequest unfortunately does not support the MLSD command, which is the only portable way to retrieve directory listing with file attributes in FTP protocol. See also Checking if object on FTP server is file or directory .

Your options are:

• Do an operation on a file name that is certain to fail for file and succeeds for directories (or vice versa). Ie you can try to download the "name". If that succeeds, it's a file, if that fails, it's a directory.
• You may be lucky and in your specific case, you can tell a file from a directory by a file name (ie all your files have an extension, while subdirectories do not)
• You use a long directory listing ( LIST command = ListDirectoryDetails method) and try to parse a server-specific listing. Many FTP servers use *nix-style listing, where you identify a directory by the d at the very beginning of the entry. But many servers use a different format. The following example uses this approach (assuming the *nix format)

Sub DownloadFtpDirectory(
        url As String, credentials As NetworkCredential, localPath As String)
    Dim listRequest As FtpWebRequest = WebRequest.Create(url)
    listRequest.Method = WebRequestMethods.Ftp.ListDirectoryDetails
    listRequest.Credentials = credentials

    Dim lines As List(Of String) = New List(Of String)

    Using listResponse As FtpWebResponse = listRequest.GetResponse(),
          listStream As Stream = listResponse.GetResponseStream(),
          listReader As StreamReader = New StreamReader(listStream)
        While Not listReader.EndOfStream
            lines.Add(listReader.ReadLine())
        End While
    End Using

    For Each line As String In lines
        Dim tokens As String() =
            line.Split(New Char() {" "}, 9, StringSplitOptions.RemoveEmptyEntries)
        Dim name As String = tokens(8)
        Dim permissions As String = tokens(0)

        Dim localFilePath As String = Path.Combine(localPath, name)
        Dim fileUrl As String = url + name

        If permissions(0) = "d" Then
            If Not Directory.Exists(localFilePath) Then
                Directory.CreateDirectory(localFilePath)
            End If
            DownloadFtpDirectory(fileUrl + "/", credentials, localFilePath)
        Else
            Dim downloadRequest As FtpWebRequest = WebRequest.Create(fileUrl)
            downloadRequest.Method = WebRequestMethods.Ftp.DownloadFile
            downloadRequest.Credentials = credentials

            Using downloadResponse As FtpWebResponse = downloadRequest.GetResponse(),
                  sourceStream As Stream = downloadResponse.GetResponseStream(),
                  targetStream As Stream = File.Create(localFilePath)
                Dim buffer As Byte() = New Byte(10240 - 1) {}
                Dim read As Integer
                Do
                    read = sourceStream.Read(buffer, 0, buffer.Length)
                    If read > 0 Then
                        targetStream.Write(buffer, 0, read)
                    End If
                Loop While read > 0
            End Using
        End If
    Next
End Sub

Use the function like:

Dim credentials As NetworkCredential = New NetworkCredential("user", "mypassword")
Dim url As String = "ftp://ftp.example.com/directory/to/download/"
DownloadFtpDirectory(url, credentials, "C:\target\directory")

---

If you want to avoid troubles with parsing the server-specific directory listing formats, use a 3rd party library that supports the MLSD command and/or parsing various LIST listing formats; and recursive downloads.

For example with WinSCP .NET assembly you can download whole directory with a single call to the Session.GetFiles :

' Setup session options
Dim SessionOptions As SessionOptions = New SessionOptions
With SessionOptions
    .Protocol = Protocol.Ftp
    .HostName = "ftp.example.com"
    .UserName = "user"
    .Password = "mypassword"
End With

Using session As Session = New Session()
    ' Connect
    session.Open(SessionOptions)

    ' Download files
    session.GetFiles("/directory/to/download/*", "C:\target\directory\*").Check()
End Using

Internally, WinSCP uses the MLSD command, if supported by the server. If not, it uses the LIST command and supports dozens of different listing formats.

The Session.GetFiles method is recursive by default.

(I'm the author of WinSCP)

Answer 2

Check out my FTP class: Its pretty straight forward.

Take a look at my FTP class, it might be exactly what you need.

Public Class FTP
        '-------------------------[BroCode]--------------------------
        '----------------------------FTP-----------------------------
        Private _credentials As System.Net.NetworkCredential
        Sub New(ByVal _FTPUser As String, ByVal _FTPPass As String)
            setCredentials(_FTPUser, _FTPPass)
        End Sub
        Public Sub UploadFile(ByVal _FileName As String, ByVal _UploadPath As String)
            Dim _FileInfo As New System.IO.FileInfo(_FileName)
            Dim _FtpWebRequest As System.Net.FtpWebRequest = CType(System.Net.FtpWebRequest.Create(New Uri(_UploadPath)), System.Net.FtpWebRequest)
            _FtpWebRequest.Credentials = _credentials
            _FtpWebRequest.KeepAlive = False
            _FtpWebRequest.Timeout = 20000
            _FtpWebRequest.Method = System.Net.WebRequestMethods.Ftp.UploadFile
            _FtpWebRequest.UseBinary = True
            _FtpWebRequest.ContentLength = _FileInfo.Length
            Dim buffLength As Integer = 2048
            Dim buff(buffLength - 1) As Byte
            Dim _FileStream As System.IO.FileStream = _FileInfo.OpenRead()
            Try
                Dim _Stream As System.IO.Stream = _FtpWebRequest.GetRequestStream()
                Dim contentLen As Integer = _FileStream.Read(buff, 0, buffLength)
                Do While contentLen <> 0
                    _Stream.Write(buff, 0, contentLen)
                    contentLen = _FileStream.Read(buff, 0, buffLength)
                Loop
                _Stream.Close()
                _Stream.Dispose()
                _FileStream.Close()
                _FileStream.Dispose()
            Catch ex As Exception
                MessageBox.Show(ex.Message, "Upload Error: ", MessageBoxButtons.OK, MessageBoxIcon.Error)
            End Try
        End Sub
        Public Sub DownloadFile(ByVal _FileName As String, ByVal _ftpDownloadPath As String)
            Try
                Dim _request As System.Net.FtpWebRequest = System.Net.WebRequest.Create(_ftpDownloadPath)
                _request.KeepAlive = False
                _request.Method = System.Net.WebRequestMethods.Ftp.DownloadFile
                _request.Credentials = _credentials
                Dim _response As System.Net.FtpWebResponse = _request.GetResponse()
                Dim responseStream As System.IO.Stream = _response.GetResponseStream()
                Dim fs As New System.IO.FileStream(_FileName, System.IO.FileMode.Create)
                responseStream.CopyTo(fs)
                responseStream.Close()
                _response.Close()
            Catch ex As Exception
                MessageBox.Show(ex.Message, "Download Error: ", MessageBoxButtons.OK, MessageBoxIcon.Error)
            End Try
        End Sub
        Public Function GetDirectory(ByVal _ftpPath As String) As List(Of String)
            Dim ret As New List(Of String)
            Try
                Dim _request As System.Net.FtpWebRequest = System.Net.WebRequest.Create(_ftpPath)
                _request.KeepAlive = False
                _request.Method = System.Net.WebRequestMethods.Ftp.ListDirectoryDetails
                _request.Credentials = _credentials
                Dim _response As System.Net.FtpWebResponse = _request.GetResponse()
                Dim responseStream As System.IO.Stream = _response.GetResponseStream()
                Dim _reader As System.IO.StreamReader = New System.IO.StreamReader(responseStream)
                Dim FileData As String = _reader.ReadToEnd
                Dim Lines() As String = FileData.Split(New String() {Environment.NewLine}, StringSplitOptions.RemoveEmptyEntries)
                For Each l As String In Lines
                    ret.Add(l)
                Next
                _reader.Close()
                _response.Close()
            Catch ex As Exception
                MessageBox.Show(ex.Message, "Directory Fetch Error: ", MessageBoxButtons.OK, MessageBoxIcon.Error)
            End Try
            Return ret
        End Function

        Private Sub setCredentials(ByVal _FTPUser As String, ByVal _FTPPass As String)
            _credentials = New System.Net.NetworkCredential(_FTPUser, _FTPPass)
        End Sub
    End Class

To initialize:

Dim ftp As New FORM.FTP("username", "password")

ftp.UploadFile("c:\file.jpeg", "ftp://domain/file.jpeg")

ftp.DownloadFile("c:\file.jpeg", "ftp://ftp://domain/file.jpeg")

Dim directory As List(Of String) = ftp.GetDirectory("ftp://ftp.domain.net/")
        ListBox1.Items.Clear()
        For Each item As String In directory
            ListBox1.Items.Add(item)
        Next