如何真正使用HttpRequestMessage，避免HttpClient PostAsync 500错误？

Question

It is supposed to fill "textbox1" with string value and get another string in textbox3. For this job, I inspected ASP form Elements in the website and checked in "Network" section the "POST" activities when I fill the form and submit it.Therefore I wrote this code for sending the post request and getting the response. But I get the error which tells me "Make sure request headers are used with "HttpRequestMessage" and "HttpResponseMessage". Before I adding the headers section I got Error 500 in console. please tell me the correct coding.

using System;
using System.Collections.Generic;
using System.Text;
using System.Threading.Tasks;
using System.Net.Http;
using System.Net.Http.Headers;



namespace WebRequest
{
    class myRequest3
    {
      
        public async static void submitpost()
        {
            using (var client = new HttpClient())
            {
                var values = new Dictionary<string, string>
        {
           {"ctl00$ScriptManager1","ctl00$content$ctl00|ctl00$content$Button4" },
           {"__EVENTTARGET","" },
           {"__EVENTARGUMENT","" },
           {"__VIEWSTATE", "/wEPDwUKLTM1MTY1NjE1Nw9kFgJmD2QWAgIJD2QWAgIDD2QWAgIDD2QWAgIBD2QWAmYPZBYCAgEPDxYCHgZfVGl0bGUFGtiz2LHbjNin2YQg2YHYudin2YQg2LPYp9iyZGRkP5ZIUYR1R/p/JMF4Ez0q2psS7pQ=    " },
           {"__VIEWSTATEGENERATOR", "CA0B0334" },
           {"__EVENTVALIDATION", "/wEWBAL6k4rsBQLt4KPyCgL84ejwCALt4KvyCh3brVZZShAeom3oNb8A0uWDpqDK" },
            {"ctl00$content$TextBox1", "hfc0Ac0TIaFPrpef5IGZJQTrCrBdm4S+" },
            {"ctl00$content$TextBox3", "" },
            {"__ASYNCPOST", "true" },
            {"ctl00$content$Button4", "ساخت سریال فعال ساز" },
        };

                var content = new FormUrlEncodedContent(values);
                client.DefaultRequestHeaders.Add("Accept","*/*");
                client.DefaultRequestHeaders.Add("Accept-Encoding","gzip, deflate");
                client.DefaultRequestHeaders.Add("Accept-Language", "en-US,en;q=0.5");
                client.DefaultRequestHeaders.Add("Cache-Control", "no-cache");
                client.DefaultRequestHeaders.Add("Connection","keep-alive");
                client.DefaultRequestHeaders.Add("Content-Length", "641");
                client.DefaultRequestHeaders.Add("Content-Type", "application/x-www-form-urlencoded; charset=utf-8");
                client.DefaultRequestHeaders.Add("Host","address");
                client.DefaultRequestHeaders.Add("Origin","http://address");
                client.DefaultRequestHeaders.Add("Referer","http://address/");
                client.DefaultRequestHeaders.Add("User-Agent", "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:79.0) Gecko/20100101 Firefox/79.0");
                client.DefaultRequestHeaders.Add("X-MicrosoftAjax","Delta=true");


                var response = await client.PostAsync("http://address/default.aspx", content);

                var responseString = await response.Content.ReadAsStringAsync();
                
                Console.WriteLine(responseString);
                

            }

            
        }



    }
}

Answer 1

There are few things you need to notice: Default request headers - some of them are predefined in content or httpclient class, so you should never do that, eg the line that is breaking your code

client.DefaultRequestHeaders.Add("Content-Type", "application/x-www-form-urlencoded; charset=utf-8");

will always give you error, since that is being done when you're generating content. So you can check it like this:

var content = new FormUrlEncodedContent(values);
var checkContentType = content.Headers.ContentType;// == "application/x-www-form-urlencoded; 

So, if we redactor your headers code a little bit with only necessary one's that's what we get:

client.DefaultRequestHeaders.Add("Accept", "*/*"); //ok
//  client.DefaultRequestHeaders.Add("Accept-Encoding", "gzip, deflate"); // unnecessary
//  client.DefaultRequestHeaders.Add("Accept-Language", "en-US,en;q=0.5"); // unnecessary
//  client.DefaultRequestHeaders.Add("Cache-Control", "no-cache"); // unnecessary
//  client.DefaultRequestHeaders.Add("Connection", "keep-alive");
//  client.DefaultRequestHeaders.Add("Content-Length", "641"); // creating content handles that - unnecessary
          
//  client.DefaultRequestHeaders.Add("Content-Type", "application/x-www-form-urlencoded; charset=utf-8"); // already set when creating content
//  client.DefaultRequestHeaders.Add("Host", "address"); // unnecessary
//  client.DefaultRequestHeaders.Add("Origin", "http://address"); // unnecessary
//  client.DefaultRequestHeaders.Add("Referer", "http://address/"); //unnecessary
client.DefaultRequestHeaders.Add("User-Agent", "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:79.0) Gecko/20100101 Firefox/79.0");
//  client.DefaultRequestHeaders.Add("X-MicrosoftAjax", "Delta=true"); // unnecessary

After I run the code now the response is 200 (ok), but server returns "0|error|500||" so you will need to check server side params/form data that are required or missing.

Answer 2

Error 500 was solved by revising dictionary values. I have entered "__VIEWSTATE" value incorrectly, besides for the first error, some of the values were unnecessary. by parsing the responseString I could reach to textbox3 value. Here is the correct code:

                var client = new HttpClient();    
                var values = new Dictionary<string, string>{
                    {"__VIEWSTATE", "/wEPDwUKLTM1MTY1NjE1Nw9kFgJmD2QWAgIJD2QWAgIDD2QWAgIDD2QWAgIBD2QWAmYPZBYCAgEPDxYCHgZfVGl0bGUFGtiz2LHbjNin2YQg2YHYudin2YQg2LPYp9iyZGRkP5ZIUYR1R/p/JMF4Ez0q2psS7pQ=" },
                    {"__EVENTVALIDATION", "/wEWBAL6k4rsBQLt4KPyCgL84ejwCALt4KvyCh3brVZZShAeom3oNb8A0uWDpqDK" },
                    {"ctl00$content$TextBox1", "hfd0Ac0TIaFPrpef5IGZJQTrCrBdm4S+" },
                    {"ctl00$content$Button4", "ساخت سریال فعال ساز" },
                };
                var content = new FormUrlEncodedContent(values);
                client.DefaultRequestHeaders.Add("Accept", "*/*");                    
                HttpResponseMessage response = await client.PostAsync("http://address/default.aspx", content);
                Thread.Sleep(50);
                string responseString = await response.Content.ReadAsStringAsync();
                Thread.Sleep(50);
                string textbox3Value= HTMLValue(responseString, "ctl00_content_TextBox3");
                Console.WriteLine("finished");    

Here is a custom code I wrote for extracting textBox3 value:

  public static string HTMLValue(string responseResult, string xElement)
        {
            int x = 0;
            int y = 0;
            string value = "";
            x = responseResult.IndexOf(xElement);
            responseResult = responseResult.Substring(x + 1);
            x = responseResult.IndexOf(">");
            y = responseResult.IndexOf("<");
            if(y-x-1>0)value = responseResult.Substring(x + 1, y - x - 1);
            return (value);
        }