VB.NET HttpWebRequest download file in chunks

Question

I'm looking for a help with following:

In my Windows Forms application, I'm downloading the file located on remote server I control. My code below:

    Public Shared Function DownloadFileWithPOST(ByVal httpPath As String, ByVal storePath As String, postdata As String) As String
    Try
        Dim theResponse As HttpWebResponse
        Dim theRequest As HttpWebRequest
        Dim postdatabytes As Byte()

        theRequest = HttpWebRequest.Create(httpPath)


        If My.Settings.ProxyURL <> "" Then
            Dim prx As New WebProxy(My.Settings.ProxyURL)
            theRequest.Proxy = prx
        End If


        '  theRequest.Timeout = My.Settings.RequestTimeout

        postdatabytes = System.Text.Encoding.UTF8.GetBytes(postdata)
        theRequest.Method = "POST"
        theRequest.ContentType = "application/x-www-form-urlencoded"
        theRequest.ContentLength = postdatabytes.Length

        Using stream = theRequest.GetRequestStream()
            stream.Write(postdatabytes, 0, postdatabytes.Length)
        End Using



        theResponse = theRequest.GetResponse


        Dim length As Double = theResponse.ContentLength




        Dim writeStream As New IO.FileStream(storePath, IO.FileMode.Create)


        Dim nRead As Integer

        Dim readBytes(4095) As Byte
        Dim bytesread As Integer = theResponse.GetResponseStream.Read(readBytes, 0, 4096)



        Do Until bytesread = 0



            'speedtimer.Start()


            nRead += bytesread



            writeStream.Write(readBytes, 0, bytesread)


            bytesread = theResponse.GetResponseStream.Read(readBytes, 0, 4096)
        Loop

        'Close the streams
        theResponse.GetResponseStream.Close()
        writeStream.Close()


        Return "OK"
    Catch ex As Exception
        Return ex.Message
    End Try
End Function

The problem is that we're getting timeout exception if the download is not completed in specified time (the commented line with timeout setting). If the download is chunked, I would see the file on drive growing on it size. However, instead of this, only when response is completely finished the file appears.

On the server side, I'm sending chunks by using response.outputstream.write method.

What to do to not get time-outed on larger files on slow connection?

Answer 1

I found, that this occurs only when you send POST request. I changed the params to be part as URL querystring and the chunked downloading now works.

    Public Shared Function DownloadFileWithQueryString(ByVal httpPath As String, ByVal storePath As String, querystring As String) As String
    Try
        Dim theResponse As HttpWebResponse
        Dim theRequest As HttpWebRequest
        Dim fullurl As String = httpPath & "?" & querystring
        theRequest = HttpWebRequest.Create(fullurl)


        If My.Settings.ProxyURL <> "" Then
            Dim prx As New WebProxy(My.Settings.ProxyURL)
            theRequest.Proxy = prx
        End If




        theResponse = theRequest.GetResponse

        Dim length As Double = theResponse.ContentLength




        Dim writeStream As New IO.FileStream(storePath, IO.FileMode.Create)


        Dim nRead As Integer

        Dim readBytes(4095) As Byte
        Dim bytesread As Integer = theResponse.GetResponseStream.Read(readBytes, 0, 4096)



        Do Until bytesread = 0



            'speedtimer.Start()


            nRead += bytesread



            writeStream.Write(readBytes, 0, bytesread)


            bytesread = theResponse.GetResponseStream.Read(readBytes, 0, 4096)
        Loop

        'Close the streams
        theResponse.GetResponseStream.Close()
        writeStream.Close()


        Return "OK"
    Catch ex As Exception
        Return ex.Message
    End Try
End Function