Get HTTP Request Headers VB.Net / Telerik Testing Framework

Question

I have a list of files that I need to download from a site that requires a login. The file will not download unless the appropriate HttpRequestHeader.Cookie is set.

If I login to the site manually and pull the headers through fiddler and pass these into the Cookie slot, the file downloads. What I'm trying to do is navigate to the site (using Telerik Testing Framework), login, read the URLs I need to download and then call the WebClient.DownloadFile.

I attempted to download the files through Telerik but was having issues once I deployed the job onto Windows Server.

After much googling, I still can't figure out how I can pull the HttpRequestHeaders through Telerik after I navigate to the page where I'm gathering the URLs - these should be the correct values I need to pass into the WebClient to download the file.

Can somebody point me in the right direction or offer an alternative?

Thanks in advance.

        Using client
            client.Credentials = New NetworkCredential("user", "pass")
            client.Headers.Add(System.Net.HttpRequestHeader.Accept, "*/*")
            client.Headers.Add(System.Net.HttpRequestHeader.AcceptEncoding, "gzip, deflate")
            client.Headers.Add(System.Net.HttpRequestHeader.AcceptLanguage, "en-US")
            client.Headers.Add(System.Net.HttpRequestHeader.UserAgent, "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; InfoPath.3; .NET4.0C; .NET4.0E)")
            client.Headers.Add(System.Net.HttpRequestHeader.Cookie, "WLP-COOKIE1=r1593517685; JSESSIONID=Sz6G1lpvwhGB6FmqN6ySPbtf2cK2fs09hTtsX2sqcSwtLL7rKzpy!-1832769242; iPlanetDirectoryPro=AQIC5wM2LY4SfczHcm5%2FeHFKpWlxpQ71jtwzwJ75uHryjzA%3D%40AAJTSQACMTAAAlMxAAIwMQ%3D%3D%23; MMISPRODPOSCsource=AQIC5wM2LY4SfczHcm5%2FeHFKpWlxpQ71jtwzwJ75uHryjzA%3D%40AAJTSQACMTAAAlMxAAIwMQ%3D%3D%23")
            client.Headers.Add(System.Net.HttpRequestHeader.Referer, "https://mysite.com")
            client.DownloadFile(url, file)
        End Using

Answer 1

I have written this function a long time ago for me and it does what you need. Try to call it for every redirect which you see in fiddler and it will save all response cookies in the global cookie object. I've auto converted this this code from c#, so maybe you need to modify it a little.

Private cookies As CookieContainer = Nothing
Public Function GetPOST(url As String, data As String) As String
    Dim httpWebRequest__1 As HttpWebRequest = Nothing
    Try
        httpWebRequest__1 = DirectCast(HttpWebRequest.Create(New Uri(url)), HttpWebRequest)

        httpWebRequest__1.Method = If([String].IsNullOrWhiteSpace(data), WebRequestMethods.Http.[Get], WebRequestMethods.Http.Post)
        httpWebRequest__1.ContentType = "application/x-www-form-urlencoded"
        httpWebRequest__1.UserAgent = _userAgent
        httpWebRequest__1.Accept = "*/*"
        httpWebRequest__1.Headers.Add(HttpRequestHeader.AcceptLanguage, "de-DE,de;q=0.8,en-US;q=0.7,en;q=0.3")
        httpWebRequest__1.Headers.Add(HttpRequestHeader.AcceptCharset, "ISO-8859-1,utf-8;q=0.7,*;q=0.3")
        httpWebRequest__1.Headers.Add(HttpRequestHeader.AcceptEncoding, "gzip,deflate,sdch")
        httpWebRequest__1.Headers.Add("X-Requested-With", "XMLHttpRequest")
        httpWebRequest__1.AutomaticDecompression = DecompressionMethods.GZip Or DecompressionMethods.Deflate

        httpWebRequest__1.Timeout = InlineAssignHelper(httpWebRequest__1.ReadWriteTimeout, 15000)
        httpWebRequest__1.CookieContainer = cookies

        If Not [String].IsNullOrWhiteSpace(data) Then
            Dim dataBytes As Byte() = ASCIIEncoding.[Default].GetBytes(data)
            httpWebRequest__1.ContentLength = dataBytes.Length
            Using stream As Stream = httpWebRequest__1.GetRequestStream()
                stream.Write(dataBytes, 0, dataBytes.Length)
            End Using
        End If

        If httpWebRequest__1.CookieContainer Is Nothing Then
            httpWebRequest__1.CookieContainer = New CookieContainer()
        End If

        Dim res As HttpWebResponse = DirectCast(httpWebRequest__1.GetResponse(), HttpWebResponse)
        Using stream As Stream = res.GetResponseStream()
            Dim str As New StreamReader(stream)
            _currentResponceLocation = res.ResponseUri.ToString()
            Return str.ReadToEnd()
        End Using
    Catch generatedExceptionName As Exception
        Return GetPOST(url, data)
    Finally
        For Each c As Cookie In httpWebRequest__1.CookieContainer.GetCookies(New Uri("https://" + httpWebRequest__1.Host))
            cookies.SetCookies(New Uri("https://" + httpWebRequest__1.Host), c.Name + "=" + c.Value)
        Next
    End Try
End Function

Answer 2

Finally stumbled upon this and it was rather simple through Telerik. I just pass the result of this function into the the cookie header for the webclient:

Public Function GetCookies(ByVal url As String) As String

    Dim cookies As String

    For Each cookie In xBrowser.Cookies.GetCookies(url)
        If String.IsNullOrEmpty(cookies) Then
            cookies &= cookie.ToString
        Else
            cookies &= "; " & cookie.ToString
        End If
    Next

    Return cookies

End Function