Trouble getting redirect trying to download a file with WebClient

Question

How can I get a file to download from an aspx page and bypass the save file dialogue?

I am attempting to download a file from a webpage and having issue. The idea is to avoid responding to the save file dialogue and just save the file straight to my computer.

The webpage requires credentials., so I am currently using a WebBrowser control to login and navigate to the page that has the button which generates the download. Then I am using code found here (the code with 16 ish upvotes, basically using InternetGetCookieEx from "wininet.dll" to extract the cookies).

Using Fiddler2, I am able to see that pressing the button generates a POST request. I mimic the request below.

My code I am trying so far:

    Dim testUri As New Uri(WebBrowser1.Url.ToString)

    request = WebRequest.Create(WebBrowser1.Url.ToString)

    Dim c = Cookies.GetUriCookieContainer(testUri)

    With request
        .Method = "POST"
        .Accept = "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8"
        .UserAgent = "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; .NET CLR 2.0.50727; .NET CLR 3.0.04506.648; .NET CLR 3.5.21022; InfoPath.2; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729; .NET4.0C; .NET4.0E)"
        .CookieContainer = Cookies.GetUriCookieContainer(testUri)
        .ContentLength = 0
        .ContentType = "application/x-www-form-urlencoded"
        .AllowAutoRedirect = True
    End With

    Dim response = request.GetResponse

    Console.WriteLine(response.ResponseUri)
    Console.WriteLine(response.ContentType)

    Dim wc As New WebClient
    wc.Headers.Add(HttpRequestHeader.Cookie, c.GetCookieHeader(response.ResponseUri))
    wc.DownloadFile(response.ResponseUri, "C:\report.xls")

Cookies is a class I wrote to hold the code in the link (below, in case it doesn't work) Imports System.Net Imports System.Text

Public Class Cookies
    <Runtime.InteropServices.DllImport("wininet.dll")> _
Private Shared Function InternetGetCookieEx(ByVal url As String, ByVal cookieName As String, _
                                     ByVal cookieData As StringBuilder, ByRef size As Integer, _
                                     ByVal dwFlags As Int32, ByVal lpReserved As IntPtr) As Boolean
    End Function

    Const InternetCookieHttponly As Int32 = &H2000

    Public Shared Function GetUriCookieContainer(ByVal uri As Uri) As CookieContainer

        Dim cookies As CookieContainer = Nothing
        Dim datasize As Integer = 8192 * 16

        Dim cookieData As StringBuilder = New StringBuilder(datasize)

        If Not InternetGetCookieEx(uri.ToString, Nothing, cookieData, datasize, InternetCookieHttponly, IntPtr.Zero) Then

            If datasize < 0 Then
                Return Nothing
            End If

            cookieData = New StringBuilder(datasize)
            If Not InternetGetCookieEx(uri.ToString, Nothing, cookieData, datasize, InternetCookieHttponly, IntPtr.Zero) Then
                Return Nothing
            End If

        End If

        If cookieData.Length > 0 Then
            cookies = New CookieContainer
            cookies.SetCookies(uri, cookieData.ToString.Replace(";", ","))

            'Console.WriteLine(cookieData.ToString.Replace(";", ","))

        End If

        Return cookies

    End Function

End Class

Setting the cookies works fine, but my actual request is not returning the data expected, it is just saving me a copy of webpage (funky, since I am trying to open it with excel. If I save it as text, I get the HTML to the whole page).

The code on the page for the button doesn't seem to point to anything. I can't find any references to the listed jQuery - I might have to mimic that?

<td align="center" id="excel_cell" noWrap="nowrap" style="width: 50px;">
    <input name="xcel" title="Send report to excel" id="xcel" style="background-image: url(../resx/images/excel.png); BORDER-BOTTOM: medium none; border-left: medium none; background-color: transparent; width: 32px; background-repeat: no-repeat; height: 32px; border-top: medium none; cursor: pointer; border-right: medium none;" type="button" jQuery1710025381725129178523="48"/>
        Text - Empty Text Node

I have tried just "clicking" the button in the webbrowser with

WebBrowser1.Document.Window.Document.GetElementById("xcel").InvokeMember("click")

But I'm trying to hide the save file dialogue from the user and automate the entire process.

Any help would be much appriciated!

EDIT : Fixed!

Solution - I needed to add to the headers, additionally, I needed to correct my POST data

    Dim postData As String = xxxx
    Dim byteArray As Byte() = Encoding.UTF8.GetBytes(postData)
    Dim dataStream As Stream = request.GetRequestStream

    dataStream.Write(byteArray, 0, byteArray.Length)
    dataStream.Close()

Answer 1

I'm assuming from the info provided so far there are no fields that need to be included in the POST. If that's the case, I recommend doing a deeper dive with Fiddler, to compare the requests your code is making with those made when manually executing in the web browser. It could be that there are other headers that you need to send that the server-side code is checking for. If there are differences, and you can refine your code until it is 100% emulating what the manual request looks like, you have a good chance of solving this through process of elimination.

You may find that for example, they are looking for the "referer" header to be set. In which case add another line to your header code:

wc.Headers.Add("Referer", yourReferringUrl);